AD- A 143  232 


'".w'?*"-  '4 


ATENT  PRATT  DIMENSION 


:<  i* U Crfut.OU  * AL  TEST  £ 


RT^OST  8-82-SONR 


Department  of  Mathematics 
University.  Of.  Illinois 
at  V rbana^Champaign 


Juno  1984 


fhis  research  va  s'sp ori sofrtd  by  the 
.^Personnel  .and  Traint-ng  Reheard.  i'lVgiJ; 
ogical  Science  Division,  Office  oi:  Nava 
ftfr  Contract:  No.  N00014-82-K-048U ;  NT,  1 


for  public  release;  distributin' 
^Reproduction  In  whole  or  in  part,  i; 
b^ifr  any  purpose  .  f  the  United  State  : 


rv  vt 

HP*-- 

P' 

f  j  ^ 

1. 111  1 .1  .si  i  o 


SLCunr.  V  Cl  AVjl  i.  '•  u*l  OI  TKh  1‘AOC  t'U,rn  Dtlt 


i^V'0:VT  B'JCl'.’lEMATION  PAGE 


UK  AD  iNSTkUCTIor.S 
nn  oi’r  roupi.i-  r;.;c,  i  oi-.m 


i.  i.l  nil!  (.  i  i- 

Fiii.il  Report  8--.S2 -f'W. 

4.  i  I  T  i.  E  /• t •  /  /,*#«  * 


*15.  COVT  ACCESSION  NO  I  RFCIPIF  N  ! ’T  i  ATAl.il  .  i.iiMin  H 

Jdl-JA*UU 


— I 


Tlif  Statistical  Assess  riant  of  Latent  Trait 
Dimensionality  in  t’.-v,  c*!;o  logical  Testing. 


7.  »' OThONf.ii 

V.’illiam  Stout 


'vet  or'  n,  roiu  *  ra  -  ,  To.tR'j 


6  ME11F  -  no.  l 


j  r;  a 


B  COS  I  i*  A  v  1  i  ■  i  G  *7  A  7  S  i‘  iv’  ’J  L  ki  s  ) 


NGOOi  4-«S2-!'.-04ot) 


».  FFRfOrlUINO  OllJ/.NlZ  AT  10‘J  J  *S'i"  AND  ADDRESS 

j  Ante  •»  fit,  i  .  u  n  i  i  rrjviit.  M‘i 

Department  of  Mathematics,  University  of  I.lli noisS  1..153M ;  KP.042-04;  HR  042-04-01 

T  Ken  T.T.-  -  r* _ _ c'  ^  . _ . 


1C.  P~O0R.’.\!  l  EM  L  N  T.  V*  ~T  A  jrl 

AREA  *  r.Cr  ;  unit  l.'L'Vlit.  fi-, 


1409  West  Green  Street 
Ur!J>aaaJ_  II.  61801 


It.  CCNTPOOU'IC  CO'Ci:  NM.IE  AND  ADDRESS 

Personnel  and  Training  Research  Program 
Office  of  Naval  Research  (Code  442PT) 

800  N.  Quincy  St  ,  Arlington,  VA  22217 


NR  100-488 


12.  REROiTT  O/V  rc 

May,  1984 
77.  nUf-  DF  r’a-.TTs 

60 


|  14.  MONITORING  AGENCY  NAMfc  b  AOORESSfl'/  dliieront  trem  Confronting  Offtcij  J  '<?  ■  l»ECU^lTy  CLAiS.  (of  .hrs  resort; 

Unclassified 


Tia.  Pi  CL  AS5lf'|CA  I'lOs  OCAS  ifi  A',,  4  C 
SCHEDULE 


7iT~USTRIC.U7ir.J4  STATEMENT  (oltl.is  Report) 

Approved  for  public  release;  distribution  unlimited 


*7.  r.l$TP.IB<JTlO;j  SI  ATGVENT  (o/  f hi  shared  entered  In  Block  20,  it  different  from  Report) 


19.  $UPrM.EME:  TA.1V  NOTES 


19.  KEY  WORDi  (Cor.t:nuo  on  reverse  %.c*t  if  ncceitery  and  identify  by  block  number) 


Item  response  theory,  latent  trait  theory,  dlmensionalitv,  unidlmensionality , 
statistical  test  of  uni dimensionality,  large  simple  theory 


L 


2Z  ACCTRA  “T  {Vnrif/ntM*  <v.  reverse  tide  li  necr enery  end  Identify  by  llock  number) 

/'Assuming  a  nonpr.rametric  item  response  theory  model,  a  large  sample  procedure, 
for  testing  the  unidinensionality  of  the  latent  ability  space  is  proposed.  * 
Under  the  assumption  of  unidimensionality,  the  asymptotic  distribution  of  the 
test  statistic  is  derived,  thereby  establishing  an  asymptotically  valid 
statistical  tost  of  unidimensionality.  A  rigorous  mathematical  definition  of 
rKmensionali ty  is  proposed  as  an  alternative  to  the  classical  item  response 
theory  definition  of  dimensionality*  This  new  definition,  while  item  response 
theory  based,  U  more  analogous  to  the  factor  analytic  notion  of  dimensionality, 

—  ,  mi  --  -  .  .v_— -J r 


DO 


I  JAM  71 


1473 


0 

_ _ 

SECURITY  CLASSIFICATION  OK  THIS  rwc  (V.htn  i'l.  m 

i 

*  dsw  i  jisk 


C/1  <  / 

^  -  Unclassified _ 

SECURITY  CL  A'jSI  Fie  A  T>0N  Or  THIS  F  <  OF ,  I'Tim  />«!<■  Frtrrri', 


\  ■  ,, 


and  as  such  is  more  congruent  with  tijo  conception  of  dimensional  itv  held 
by  applicators.  The  statistical  tost  procedure  is  shown  U>  have  asvmpi.ot  i , 
power  1  for  multi dimension'll  (in  our  sense)  tests.  Monte  Carlo  studies 
using  a  specially  written  Fortran  program, indicate  good  agreement  wife  t  he 
nominal  level  of  significance  when  uni u  'mens ional  ity  hold  -,  and  good  power- 
performance  for  examinee  sample  sizes  and  psychological  ted  lengths  often 
encountered  in  practice'..  This  adherence  to  the  nominal  level  of  signifi¬ 
cance  and  good  power  performance  is  also  the  case,  as  desired,  when  the 
items  are  "multiple  determined*',  thereby  producing  a  classical  dimension¬ 
ality  much  greater  than  1.  x  Finally,  an  analogous  test  procedure  i;s 
proposed  for  testing  H  :  d  <  3^  vs.  A  :  d  >  d^  for  fixed  ^2.  d 
denoting  dimensionality., 

June,  1984 


-  i  ■  o  ,  r  ip  ^ 


Note  concerning  the  relationship  of  enclosed  final 
report  to  recently  distributed  technical  report 


You  almost  certainly  recently  received  an  ONR  technical  report 
entitled  "A  Statistical  Procedure  for  Assessing  Test  Dimensionality,"  which 
I  wrote  in  the  fall  of  1981,  The  enclosed  Final  Report  totally  supersedes 
the  1981  document,  which  was  an  effort  to  articulate  my  early  work  on  the 
dimensionality  problem.  Hence,  you  should  concentrate  your  time  on  the 
enclosed  Final  Report,  which  carefully  describes  the  current  status  of  my 
dimensionality  research  in  its  appropriate  psychometric  context. 


Accession  For 

"ms"  GRA4I  V 

MIC  TA$ 

Unannounced  (_] 

Justification _ 

By_ — _ _ 

.Distribution/ 
Availability  Codes 
lAvail  and/or 
Diat  Special 


William  Stout 


'  >) 


SECURITY  CLASSIFICATION  OF  THIS  PAGEfWun  D«(»  Lnr.utri) 


The  Statistical  Assessment  of  Latent  Fiait 
Dimensionality  in  Psychological  Testing 

(Final  Technical  Report  of  N0Q014-82-K-O48fc) 

William  Stout 

University  of  Illinois  at  Urbnna-Champaign 

1.  Introduct i  on 

In  Lord  (1980),  Frederick  Lord  states  "There  is  a  great  need  for 
a  statistical  significance  test  for  the  unid  iuensiona ■  i.  ty  of  <i  r<  t  of  u.-.i 
items."  Indeed,  it  is  an  important  problem  to  he  able  to  determine  whether 
a  test  that  purports  to  measure  the  level  of  a  certain  ability  is  in  reality 
significantly  contaminated  by  the  varying  levels  of  one  of  more  other  abilities 
displayed  by  examinees  taking  the  test.  For  example,  is  a  test  of  math emeu ictil 
ability  contaminated  by  varying  levels  of  verbal  ability  displayed  by  subjects 
taking  the  test?  As  a  second  example,  it  is  of  serious  concern  in  Canada  that 
performance  on  standardized  ability  tests  not  be  influenced  by  varying  examiner 
of  familiarity  with  the  French  Canadian  or  Anglo-Canadian  cluture  (sco  Sarrazir  (193 
Further,  the  standard  item  response  theory  (IRT)  methodology  (e.g,,  LOCIST) 
is  predicated  upon  the  assumption  of  unidimensionality.  Because  of  the  large 
number  of  private  and  governmental  organizations  routinely  using  tests  to 
screen  people  by  assessing  their  levels  of  various  aptitudes  or  abilities,  tie 
problem  of  assessing  the  dimensionality  (especially  the  unidimensionality)  of  a 
test  is  of  great  importance.  Now,  with  the  imminent  use  of  computerized  adap¬ 
tive  testing  by  the  U.  S.  Armed  Forces  (a  test  setting  fov  which  any  two  examine 
taking  the  same  test  will  in  general  be  administered  non-identical  sets  of  items 
the  issue  of  dimensionality  becomes  particularly  critical. 

The  basic  objective  of  the  project  was  to  develop  a  practicable, 
theoretically  based  statistical  procedure  for  testing  for  the  unidinens ionai i t; 


2 


of  a  test.  To  this  end,  the  following  objectives  have  been  attained: 

(1)  The  development  of  an  intuitively  plausible  statistical  method  for 
testing  for  unidimensionality. 

(2)  The  (rather  delicate)  derivation  of  the  asymptotic  distribution  of  the 
proposed  test  statistic  (of  (1))  under  the  assumptions  of  unidimensionality 
and  reasonable  regularity  conditions,  thereby  establishing  an  asymptotic 
level  a  test  of  unidimensionality. 

(3)  The  development  of  a  rigorous  mathematical  definition  of  test  dimension¬ 
ality  that  is  item  response  theory  based  but  yet  is  consistent  with  the 
classical  factor  analytic  notion  of  dimensionality  and  also  is  not  adversely 
influenced  by  the  inherent  multidimensionality  of  individual  test  items. 

(4)  The  establishment  of  asymptotic  power  one  for  the  statistical  test  when¬ 
ever  unidimensionality  in  the  sense  of  the  definition  referred  to  in  (3)  fails 
to  hold. 

(5)  The  writing  of  a  FORTRAN  program  to  simulate  the  use  of  the  statistical 
test  procedure  for  the  case  of  a  unidimensional  test  and  for  the  case  of  a 
two  dimensional  test.  Here  the  two  dimensional  test  was  modeled  both  with  and 
without  multiply  determined  (i.e.,  multidimensional)  test  items. 

(6)  The  conducting  of  preliminary  Monte  Carlo  studies  to  assess  the  practic- 
ibility  of  the  proposed  test  of  unidimensionality  for  realistic  test  lengths 
and  examinee  population  sizes,  these  results  indicating  reasonable  faithfulness 
to  the  prescribed  level  of  significance,  even  in  the  presence  of  multidimens ion. 
items,  and  indicating  reasonable  power. 

(7)  The  development  of  an  analogous  test  to  that  of  unidimensionality  in  order 
to  test  (letting  d  denote  dimensionality) 

HQ  :  d  <  dQ  vs  :  d  >  d^ 

where  dp  is  a  fixed  integer  satisfying  d^  >_  2. 

The  remainder  of  the  report  is  devoted  to  a  description  of  the  attainment 
of  these  objectives  together  with  certain  prerequisite  background  information. 
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2 .  A  careful  statement  of  tin*  Item  res[>ot  e  theory  model. 

The  mathematical  definition  of  dimensionality  and  the  mathematical 
derivation  of  the  large  sample  theory  for  the  proposed  test  statistic  both 
require  a  careful  statement  of  the  assumptions  of  the  item  response  theory 

model: 

Consider  sampling  J  examinees  from  a  population  and  administering 

a  test  consisting  of  N  items  to  each  sampled  examinee.  Suppose  each  item 

Is  scored  correct  or  incorrect.  (1  for  correct  and  0  for  incorrect). 

Suppose  that  associated  with  each  examinee  is  a  vector-valued 

variable  8  =  (0,,0„ . 0.)  (the  "ability"  of  the  examinee)  that  deter- 

—  i  l  a 

mines  the  probability  of  a  correct  response  for  each  item  administered  to 
the  examinee.  For  each  sampled  examinee  j.  binary  random  variables 
{U^j }  are  observed,  where  i  is  the  item  index.  The  {U^}  are  the 
observed  or  manifest  variables  referred  to  in  the  discussion  of  latent 
structure  models.  "Item  characteristic  curves"  are  defined  by 

(2.1)  Pi(0)  -  P[Uij  =  1  |  Oj  -  0]  =  1  -  PIU^  -  0  |  gj  -  0], 

the  probability  of  a  correct  response  to  item  i,  given  that  the  jth 
sampled  examinee  has  ability  £.  Here  it  is  assumed  that  examinees  are 
randomly  sampled  from  an  infinite  population,  thereby  inducing  a  probability 
distribution  on  the  £  ability  space  with  associated  random  vector  0. 

Here  we  let  denote  the  random  vector  of  abilities  for  J 

sampled  examinees.  Throughout,  0  is  assumed  to  be  a  continuous  random 
vector  with  density  denoted  by  f(_6).  L  denotes  the  domain  of  f(e)  and 
is  a  subset  of  d  dimensional  Euclidean  space. 
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Item  response  theory  is  based  upon  the  fundamental  assumption  of 
local  independence,  which  is  assumed  throughout  this  report.  The  intuitive 
idea  is  that,  conditional  on  knowing  the  ability  of  an  examinee,  the  item 
responses  are  independent  of  each  other.  That  is,  for  example,  the  dependence 
between  and  1^,^  for  i  t  i*  is  entirely  explained  ("mediated")  by 

knowing  the  value  of  Formally,  local  independence  is  said  to  hold 

provided 

(2.2)  P(un-  vu21  -  u2 . uN1  ■=  uN  I  -  61  -  IT  P[uu  -  |  -  0] 

for  all  0  £  L  and  each  choice  of  u^  =  0  or  1.  Recall  that  h’  denotes 
the  total  number  of  items. 

It  is  also  necessary  to  assume  throughout  two  assumptions  about 
the  sampling  of  examinees  and  the  independence  of  information  provided  by 
different  examinees.  It  is  assumed  throughout  that 


(2.3) 


consists  of  independent  identically  distributed  (iid)  random  vectors.  That 
is,  the  {©j}  are  assumed  to  consist  of  a  random  sample. 


Now  let  ■  (U^j  ,1^  i . ..  ,U^) *  denoting  the  test  performance- 
It  is  assumed  that  knowing  each  examinee's  _9  produces  a  sort  of  conditional 
independence:  That  is,  it  is  assumed  throughout  that 


(2.4)  PtU^  «  ux*U2  “  =  -J  1-1  =  — 1»— 2  =  -2 . 

In  other  words,  the  responses  of  different  units  are,  conditional  on  knowing 
their  latent  values,  independent  of  each  other. 
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For  some  purposes  the  class leal  definition  of  d imenslona  1  j  r  v  used  in  I  tem 
response  theory  will  be  used  in  this  report  (sec.  Lord  and  Novick  (1968),  pp.  359-362 
for  a  good  discussion  of  this).  This  amounts  to  stating  that  d  Is  the 
correct  dimensionality  provided  the  joint  distribution  of  the  observed  random 
variables  {U^,  1  £  i  £  N,  1  <_  j  £  J}  is  representable  in  terms  of  a  d 
dimensional  distribution  of  1.1. d.  and  identical  (in  j)  conditional 

distributions  of  JJ^,  given  © j »  with  local  independence  and  (2.  A)  assumed, 
and  moreover  that  such  a  representation  is  impossible  for  d'  <  d.  Here  of 
course  by  ©  being  d  dimensional  is  simply  meant  (since  0  is 
assumed  to  be  a  random  vector  of  continuous  type)  that  the  density  f((j) 
of  ©  has  d  dimensional  Euclidean  space  as  its  domain. 

It  will  be  assumed  throughout  the  report,  that, denot  ing  here  the  classical 
dimensionality  by  d,  whenever  the  joint  distribution  of  the  {U^}  is 
modeled  In  terms  of  (identically  distributed)  latent  variables  {©  }  that 
the  distribution  of  ©  is  in  fact  d  dimensional.  Such  a  remark  is 


necessary,  since,  of  course,  it  is  always  possible  to  use  a  distribution 
for  ®  that  has  dimensionality  larger  than  d. 


•V  X  1  'N  i'v  "W  -*  -» 
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3.  A  statistical  approach  to  the  assessment  of  latent  structure  uni  dimensional Ity. 

As  stated  in  Section  1,  the  main  object  of  the  project  was  to  develop 
a  statistical  method  for  assessing  the  choice  of  dimensionality  d.  The 
dimensionality  test  of  most  importance  in  mental  test  theory  and  likely  in 
most  other  settings  too  is  the  test  of  unidimensionality: 

(3.1)  H  :  d  *=  1  vs  A  :  d  >  1. 

Our  hypothesis  testing  procedure  to  investigate  (3.1)  is  now  broken  into 
steps.  By  "score"  is  always  meant  proportion  correct. 

Steps  of  the  procedure: 

Step  1:  Split  the  N  test  items  into  a  long  subtest  of  length  n  and  a 
short  subtest  of  length  M.  These  two  subtests  are  called  respectively: 
the  partitioning  subtest  (length  n)  and  the  assessment  subtest  (length  M) . 

Observations  taken:  (U^  :  i  =  l,2,...,n+M;  j  =  1,2 . j}. 

(Typical  values  in  mental  test  .applications:  J  =  1000,  N  =  75). 

Step  2:  Partition  the  unit  interval  [0,1]  into  subintervals  {A^”\  1  £  k  <.  K  } 

•v  n 

of  equal  length.  E.g.,  typical  interval:  [0.48,  0.52),  in  which  case  there 
are  25  intervals. 

j 

Step  3:  Assign  examinees  to  partition  subintervals  according  to  their 
scores  on  the  partitioning  subtest.  E.g.,  if  an  examinee's  partitioning 
subtest  score  is  0.51  and  if  0.04  is  the  interval  length  then  the  examinee 
is  assigned  to  the  interval  [0.48,  0.52).  Let  be  the  number  of 

examinees  assigned  to  the  kth  interval  of  the  nth  partition,  the  "nth 
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partition"  being  the  partition  associated  with  a  partitioning  subtest  of 

length  n. 


Step  4:  (Construction  of  the  test  statistic) .  Resubscript  the  U  for 

n+1  <  i  <  n+M  such  that  U...  =  Uj??  indicates  the  correctness  of  the 

response  of  examinee  j  of  subinterval  k  of  partition  n  (1  ^  j  j5°^) 

K 

to  item  i.  Note  that  J  =  ^ 


(n) 


In  each  subinterval  of 
the  partition  that  contains  enough  examinees  (according  to  some  convention 


set  by  the  statistical  user),  the  difference  of  two  variance  estimates  is 

computed.  From  this  point  on,  Kn  will  denote  the  number  of  such  intervals 

rather  than  the  number  of  intervals  of  the  partition  and  A^n\ . . .  ,A^n^  will 

n 

denote  the  subintervals  containing  enough  examinees.  Computation  of 
a2 

Oy  .  »  the  first  variance  estimate  for  interval  k:  Let 
-it* - — - - — 

y(k)  £n+m  U  /ft 

j  ^i*=n+l  Uijk/n 

the  assessment  subtest  score  of  the  j  th  examinee  of  interval  k  on  the 
assessment  subtest.  (E.g.,  j  *  2  denotes  the  second  examinee  among  the 
examinees  assigned  to  interval  k  of  the  nth  partition.)  Let 

T  (n) 

(3.2)  Y(k)  =  Z^  Yj(k)/J^n), 

the  average  of  examinee  assessment  subtest  scores  for  interval  k.  Let 


,(n) 


r(k)  v(kK2._(n) 


(3.3)  Syfk  "  Ej=!  (Yw-Ywr/Jk  . 


Computation  of  Op  the  second  variance  estimate  for  interval  k: 


Let 


(3.4) 


;(k)  .  yk 
p.  Z 


t(n) 

,  ,  u. .. /J^n) 

j*l  ijk  k 


3P.k  -  O  Mk)  (1  '  ^k>)/H2' 


(3.5) 
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The  test  procedure  is  then  to  reject  H  if 


K 


y  n  (o^  -  ) 

i«k=l  '  Y,k  P,ky 


appropriately  normalized,  is  sufficiently  large. 


Normalization:  Let 


,(n) 

ck  ,v(k)  n(kK4/T(n) 
£j=l  ;  /Jk 


(3.6) 


VM  A(k) 
4i=l  Pi 


(1 


(1  -  2p 


(kh2 
i  ' 


Finally,  let 
(3.7)*  T  =  Tn 


Then,  the  procedure  is  to  reject  the  null  hypothesis  of  (4.1)  provided 
(3.8)  T  >  Z^, 

where  Z^  is  the  upper  100 (1-a)  percentile  value  for  a  standard  normal 
distribution. 

Remark.  The  selection  of  which  items  are  assigned  to  the  partitioning  sub¬ 
test  and  which  items  will  be  assigned  to  the  assessment  subtest  is  related 
to  the  power  of  the  test  and  will  be  discussed  in  Section  8. 

*See  page  30  for  a  modification  of  T  that  may  have  better  small  sample 

properties. 
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It  seems  useful  to  have  an  intuitive  understanding  of  the  procedure 

given  by  (3.8).  For  this  purpose  let  K  =  1  and  ignore  the  scaling  provided 

by  the  denominator  in  (3.7).  That  is,  think  of  the  statistical  procedure  as 

rejecting  the  hypothesis  of  unidimensionality  when  o  -  o  is  "large"  (k=l) 

X  ,  1  r  ,  1 

Recall  from  classical  test  theory  that  for  a  test  of  length  M  with  Y 
denoting  proportion  right  and  a  denoting  the  probability  that  tl  randomly 
selected  examinee  gets  item  m  right,  that  the  Kuder-Richardson  formula-20 
coefficient  P2q  is  defined  by  (e.g.,  see  Lord  (1980),  p.  8) 


M 


20 


M  -  1 


1  - 


a  (1  -  a  )/M2 
Lm=  1  in  m 


(the  division  by  M  occurs  because  Y  denotes  proportion  correct  rather 
than  number  correct).  Note  that 


(3.9)  P2Q  -  0  if  and  only  if  O2  -  -  aJ/M2  -  0. 

Of  course  P2q  “0  is  to  be  psychometrically  interpreted  (up  to  an  approximation) 

as  the  fact  that  Y  has  no  reliability  at  all.  Although  unrelated  to  the 

author's  process  of  discovery  of  his  unidimensionality  test,  (3.9)  can  be 

nicely  used  to  give  a  simple  psychometric  interpretation  and  justification 

for  the  test  procedure  (3.8):  In  the  present  test  setting,  0^  ^  "  °p  ^  is 
2  M  2 

an  estimate  of  -  ^jn_iam^  ~  am)/M  *  m  indexing  the  M  items  of  the 

2 

assessment  subtest.  Here  a  and  cl  are  computed  for  the  "population" 

m  i 

of  examinees  assigned  to  subinterval  1  by  the  assessment  subtest.  Thus 
a2  *2 

ov  ,  -  CL  -  "large"  is  evidence  that  p9,.  is  greater  than  0.  Thus, 

X)1  zu 

the  procedure  rejects  the  hypothesis  Hq  of  unidimensionality  if  and  only  if 
the  assessment  subtest  for  subinterval  1  examinees  shows  statistical  evidence 
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of  having  some  reliability.  That  is,  Hq  is  rejected  if  and  only  if  there 
is  statistical  evidence  that  the  assessment  subtest  provides  information 
about  examinee  "true  score"  (or  latent  ability  from  the  item  response 
theory  viewpoint)  beyond  that  provided  by  the  knowledge  of  which  partitioning 
subinterval  an  examinee  is  assigned  to  by  the  partitioning  subteut.  And,  the 
assessment  subtest  can  only  provide  such  information  (except  for  negligible 
finite  sample  error)  provided  there  _is  more  than  one  dimension  being  measured. 
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4.  The  large  sample  theory  when  d  =  1  . 

In  this  section  the  classical  conception  of  dimensionality  described 

at  the  end  of  Section  2  will  be  used.  The  main  purpose  of  this  section  is 

to  present  the  precise  conditions  under  which  T,  as  defined  by  (3.7),  can 

be  shown  to  have  asymptotically  a  standard  normal  (i.e.,  N(0,1))  distribution. 

It  is  however  easy  to  see  several  reasons  why  one  might  also  wish 

to  have  the  large  sample  distribution  of  the  Sv  v  “  suitably  stan- 

dardized,  for  each  subinterval  k.  First,  in  some  applications  K^,  the 

number  of  subintervals  used  in  the  construction  of  the  test  statistic  may  be 

rather  small  and  hence  the  large  sample  theory  for  T  fail  to  provide  a 

good  approximation  for  the  actual  distribution  of  T.  In  this  case,  having 

«2  ~2 

the  large  sample  distribution  for  each  °y  k  “  °p  ^  ln  T  then,  using  the 

2  a  2 

independence  of  the  3  -  o.  ,  ,  yields  a  large  sample  theory  for  T 

* » K  r )  K  i 

when  K  is  small.  Second,  one  may  wish  to  construct  a  new  test  statistic 
n 

for  unidimensionality  still  based  on  the  summands  of  T,  but  with  the  new 
test  statistic  having  a  different  formula  from  that  of  T.  Finally,  and 
perhaps  most  important,  one  may  be  interested  in  assessing  the  contribution 
to  lack  of  dimensionality  resulting  from  only  a  portion  of  the  latent  ability  spac 
That  is,  one  may  wish  to  construct  a  test  statistic  based  upon  only  certain 
subintervals  of  the  partition,  indeed  possibly  even  using  only  one  subinter¬ 
val.  Because  of  the  above,  it  is  desirable  to  obtain  the  large  sample 

a2  a  2 

distribution  of  each  °y  k  ”  °P  k  as  t*ie  num^er  test  items  and  examinees 

each  become  suitably  large.  Convergence  in  law  (i.e.,  in  distribution) 

L  2 

will  be  denoted  throughout  by  — N(y,o  )  denotes  a  normal  distribution 

2 

with  mean  \l  and  variance  0  ,  Z  always  denotes  a  N(0,1)  random  variable. 
For  example,  the  statement  that  converges  in  law  to  a  standard  normal 

random  variable  would  be  denoted  by 


The  two  basic  large  sample  theorems  of  this  section  are  based  on  a  set 
of  assumptions,  which  are  now  stated,  together  with  a  brief  description 
of  the  practical  interpretation  and  reasonableness  of  each  assumption. 

First  it  is  assumed  that 

(4.1)  d  -  1 

in  the  classical  sense  of  dimensionality.  That  is,  the  large  sample  distri- 

*2  a2 

butions  of  the  a.,  ,  -  0n  ,  and  T  are  stated  in  this  section  under  the 
Y,k  P,k 

assumption  that  the  hypothesis  H  :  d  =  1  is  true,  true  in  the  classical 
sense  of  dimensionality. 

It  is  assumed  that 

(4.2)  min  •*■<*>  as  n  -*•  00 . 

l<k<K  r 

- n 

This  simply  amounts  to  excluding  pa^tl  Hon  subintervals  that  contain  too  few 
examinees.  In  practice,  some  suitable  convention  would  be  used,  such  as 

j(n)  >  20  for  each  i  <  k  <  K  . 
k  -  -  —  n 

The  number  of  items  M  on  the  assessment  subtest  is  assumed  to 
be  fixed  as  n,  the  number  of  items  on  the  partitioning  subtest,  approaches 
».  This  means,  in  effect,  that  the  number  of  items  on  the  partitioning 
subtest  must  be  small  compared  to  the  number  of  items  on  the  assessment 
subtest.  This  amounts  to  a  genuine  restriction  in  the  design  of  the 
statistical  test  in  that  our  theory  simply  does  not  support  taking  M  »  n  *  38 
when  N  *  76  for  example.  However,  if  M  =  5,  n  =  71  produces  good  results 
in  terms  of  attaining  a  reasonably  powerful  test  with  the  actual  level  of 
significance  being  close  to  the  nominal  level  of  significance,  then  the 
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restriction  becomes  non-essential.  And,  indeed,  preliminary  Monte  Carlo 
studies  support  the  preceding  statement.  (See  Section  7.) 

It  is  assumed  that  the  M  assessment  items  are  the  same  for  every 
n.  This  is  merely  a  virtual  restriction,  having  no  "real  world"  content 
since  in  fact  in  practice  one  applies  the  procedure  for  a  particular  set 
of  M  items  and  only  one  choice  of  n.  Although  a  double  subscript  nota¬ 
tion  would  avoid  notational  inconsistency,  we  nonetheless  denote  the. item 
characteristic  curves  (ICC's)  of  these  fixed  M  items  by 

<4-3)  » ^>n+2(®) . p„+«(0);  <  9  <  “• 

Now,  the  ICC's  for  the  items  of  the  partitioning  subtest  are  denoted  by 

(4. A)  Pj(e),P2(0),...,Pn(0), 

with  the  understanding  that  P^fO)  does  not  necessarily  denote  the  same 
ICC  on  the  n'th  partitioning  subtest  as  it  does  on  the  nth  partitioning 
subtest  for  n  i  n'.  Although  assuming  that  P^(0)  is  the  same  for  all 
choices  of  partition  subtest  length  n  1  would  only  be  a  virtual  rather 
than  a  practical  restriction,  nonetheless  we  do  not  need  to  make  this 
restriction. 

Certain  assumptions  are  made  about  the  form  of  the  ICC's.  First  each 
ICC  is  assumed  to  be  continuous  and  differentiable  in  0  and  strictly  increasing  in 
Clearly  this  entirely  reasonable  restriction  needs  no  explanation  nor 
defense. 

It  is  assumed  that  there  exists  an  interval  (a,b)  such  that  for 
some  likely  small  number  £  satisfying  h  >  e  >  0  that 
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(4.5) 

P^(0)  >_  c 

for  a  <  0  b, 

1  <  i  <  N, 

(4.6) 

Pj(0)  £  C 

for  all  0,  1  <_ 

i  <  N,  some  (possibly  large)  C, 

and 

(4.7) 

e  <  Pi(0)(l 

-  P^O))  <  1  -  £ 

for  a  <  0  <  b,  1  <.  i  <  N. 

Here  (4.5) 

merely  states  that 

there  is  some  ability  range  over 

which 

items  are  uniformly  (both  as  0  varies  and  items  change)  sensitive  to  ability 
differences.  This  merely  amounts  to  discarding  items,  as  test  constructors 
do,  that  are  not  sufficiently  discriminating  or  are  too  easy  or  too  difficult. 
This  practice  of  discarding  overly  easy  and  difficult  items  also  justifies 

(4.7) ,  which  amounts  to  not  allowing  items  that  are  too  easy  or  too  difficult. 

(4.6)  of  course  states  that,  even  locally  in  0,  none  of  the  items  are 

allowed  to  be  too  discriminating.  It  is  of  course  impossible  in  virtually 

all  test  settings  to  constrict  highly  discriminating  items,  even  if  sometimes 

desirable  to  do  so.  As  Lord  and  Novick  (1968,  p.  379)  state,  "We  might  note 

that  it  is  rare  to  find  values  of  a?  (slope)  as  large  as  2  in  aptitude 

b 

and  achievement  testing." 

Let  P.(-°°)  »  lira  P.(8),  Then  let  n  *  sup  P.(-°°),  It  is  assumed 
1  0->-°o  i>l 

that  each  subinterval  of  the  partition  included  in  the  construction  of  T 
lies  in  the  interval  (n,l].  This  is  merely  a  technical  restriction  that 

will  cause  no  trouble  in  practice;  and  indeed,  if  needed,  this  restriction 

could  be  weakened.  Typically,  ri  =  0,  3/4 ,  or  1/5  is  assumed,  based  upon 

various  assumptions  about  "guessing." 

It  is  required  that 

(4.8)  J*n)K  <  Cn2 

k  n 
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for  all  k,  n  for  some  C  in  order  to  obtain  the  distribution  of  T. 

A2  *2 

If  one  is  merely  interested  in  the  distribution  of  each  of  the  ov  ,  -  a 

I  jK  1 

separately  then  (4.8)  is  weakened  to 


(4.9)  j£n)  <  Cn2 


for  all  k,  n  for  some  C.  These  two  assumptions  restrict  the  number  of 

examinees  per  subinterval  and  in  the  case  of  the  distribution  of  T,  (4.8)  restr 

the  total  number  of  intervals  contributing  to  T.  This  is  a  genuine 

restriction,  but,  fortunately,  allowing  'K  to  be  the  order  of  n  seems 

reasonable  for  most  applications.  E.g. ,  if  the  partitioning  subtest  of  a  test  consist 

of  100  items,  then  •=  30  for  each  k,  K  =30  is  such  that  K  is 

n  n  k  n 

2  A 

"small"  compared  to  n  ■  10  .  Preliminary  Monte  Carlo  studies  again  seem 

to  support  the  accuracy  of  the  large  sample  approximation  of  T  for  reason- 

Kn  (n) 

able  choices  of  n  and  ,  the  number  of  test  examinees  (ignoring 

discarded  intervals  here) . 

The  following  assumption  is  needed  for  highly  technical  reasons. 

It  could  undoubtedly  be  removed,  at  the  expense  of  a  rather  complicated 

~2  A2 

answer  for  the  large  sample  distributions  of  o  ,  -  o  .  and  T.  It  is 

I j  K  ij  K 

assumed  that  there  exists  some  (possibly  small)  e  >  0  such  that  for  all  k,  n, 
and  each  of  the  M  assessment  items  (indexed  by  i), 


(4.10)  |Pi(6^n))  -  H\  >  E 

where  6 is  defined  by,  letting  Y 

partitioning  subtest, 


denote  proportion  correct  on  the 


E[Y  |  0  =  *  midpoint  of  . 
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Here  E[X  |  A]  denotes  the  expectation  of  a  random  variable  X,  given 
that  event  A  has  occurred.  (4.10)  requires  that  partitioning  intervals 
corresponding  to  examinees  expected  to  get  "approximately"  half  right  on 
any  item  of  the  assessment  test  should  be  removed.  Since  the  M  items 
are  fixed  and  £  in  (4.10)  can  be  as  small  as  desired,  (4.10)  will  always 
be  true  in  practice.  But  nonetheless  some  Monte  Carlo  analysis  here 
should  help  support  the  author's  intuition  that  the  practical  impact  of 
ignoring  (4.10)  in  practice  should  be  minimal.  Alternatively  one  could 
merely  throw  out  those  intervals  A producing  pj^'s  exceedingly 
close  to  %  for  any  i  =  n+l,...,n+M.  The  mathematically  interested 
reader  is  invited  to  consult  p.  119  Serf  ling  (1980)  for  the  need  of  an 
assumption  such  as  (4.10). 

This  completes  the  list  and  discussion  of  the  assumptions  required 
to  obtain  the  large  sample  theory  when  d  =  1.  In  summary,  all  the  assump¬ 
tions  either  seem  quite  reasonable  or  at  least,  in  some  cases,  not  unduly 
restrictive.  Of  course,  Monte  .Carlo  studies  and  the  use  of  the  procedure  in 
actual  mental  test  data  are  needed  to  verify  this  opinion  (see  Section  7). 

Now  the  two  basic  large  sample  results  when  d  =  1  are  stated. 


Theorem  4,1.  Suppose  (4.1),  (4.2),  M  fixed  assessment  items,  continuous 
and  differentiable  ICC's,  (4.5),  (4.6),  (4.7),  all  included  subintervals 
in  (n,l)»  (4.9),  and  (4.10).  Then  (see  (3.7) ),  for  any  choice  of  integers 


k  H  k 


(4.11) 


<o 


Y,k 


-2 

o 


>4(n) 


P,k'  ~k 


ZnX(y)-sS,k,7WT 

vt 


n  w, 


I 


where  the  rate  of  convergence  of  the  distribution  function  of  Z lo  a 
N(0,1)  distribution  function  does  not  depend  on  the  sequence  {k^} . 

Theorem  A,  2.  Suppose  (A.l),  (A. 2),  M  fixed  assessment  items,  continuous 
and  differentiable  ICC's,  (A. 5),  (A. 6),  (A. 7),  all  included  subintervals 
in  (n » 1 ] ,  (A. 8),  and  (A. 10).  Then  (see  3.7), 


(A. 12)  T 


Z  as  n  -*•  00 . 


Remark.  The  denominator  of  (A.  11)  has  a  rather  tidy  matrix  representation: 

Let,  suppressing  k,  and  writing  p  for  p,^  of  (3. A)  for  convenience, 

i  i+n 

D  -  (2p^-l, 2p2~l| • .  • »  2p^-l,  1) 


PjU-p-l) 


P20-P2> 


P1(l-P1)(l-2p1) 

P2^1_P2^  ^1_2P2^ 


P1(1”P1) (l“2p1) 


...PM^-Pyj) 


p2(i-p2)d"2p2)...pM(i-pM)(i-29M)  m4^4(y)  -  a4)  1 


Then,  it  is  trivial  to  see' that 


A  ^  A  A  A  1  AAA  jl 

(A. 13)  (M4(y4(Y)  -  0*>k)  -  =  (DZD' ) 2 , 


This  may  be  useful  both  for  interpretation  and  computer  programming  purposes, 
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Proofs.  The  proofs  of  these  two  results  arc  lengthy,  delicate,  and 
make  use  of  sophisticated  results  from  mathematical  probability  theory. 

Their  presentation  here  seems  inappropriate.  The  interested  reader  is 
referred  to  Appendix  I  for  a  complete  presentation  of  the  proofs.  However, 
a  brief  sketch  of  the  highlights  of  the  proofs  seems  appropriate.  We 
will  consider  Theorem  4.1’s  proof  only,  since  that  of  Theorem  4.2  is 
rather  similar. 

Basically,  the  proof  is  based  upon  the  central  limit  theorem 

holding  for  sums  of  independent  finite  variance  random  variables,  the 

summands  suitably  similar  in  magnitude  (see  e.g.,  the  Lindeberg-Feller 

theorem  as  stated  for  triangular  arrays  of  random  variables  in  Chung  (1974), 

p.  205).  It  is  also  based  on  a  multivariate  version  of  the  "6  method", 

the  local  linearization  technique  based  on  multivariate  Taylor  series 

expansion  that  is  so  useful  in  large  sample  theory  (see  Serf ling  (1980), 

p.  122),  Further,  the  random  denominator  in  (4.11)  is  permitted  by  an 

application  of  a  Slutsky  type  argument  (see  Serf ling  (1980),  p.  19). 

Finally,  a  very  delicate  question  about  the  asymptotic  expectation  of 

/\  2  *2 

Ov  .  -  Op  ,  is  dealt  with  by  means  of  a  conditional  tail  probability 
estimate  for  the  random  ability  0,  given  that  Y  (the  partitioning 
score)  G  This  conditional  probability  estimate  follows  in  a  manner 

similar  to  the  classical  exponential  probability  bounds  for  sums  of  independent 
random  variables  (see,  e.g..  Stout  (1974),  Section  5.2)  and  is  the  most 
delicate  aspect  of  the  proof. 
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5 .  The  concept:  of  latent  trait  dimensional itv 

The  purpose  of  this  section  is  to  propose  a  concept  of  dimen¬ 
sionality  distinct  from  the  classical  item  response  theory  definition. 

Recall  that  the  classical  def  inition  amounts  to  stating  that  k  is  the  correct 
dimensionality  provided  the  joint  distribution  of  the  observed  random 
variables  {U^,  1  £  i  N,  1  £  j  £  J}  is  representable  in  terms  of  a 
k  dimensional  distribution  of  iid  0j  and  identical  (in  j)  conditional 
distributions  of  £ ^ ,  given  0  ,  with  local  independence  and  (3.4) 
assumed,  and  moreover  that  such  a  representation  is  impossible  for 
k'  <  k. 

Although  mathematically  appealing,  this  definition  is  rather 
inappropriate  for  assessing  the  power  of  tests  of  dimensionality  in  the 
case  of  mental  test  theory.  This  is  because  in  mental  testing,  individual 
test  items  clearly  have  multiple  determinants  of  their  respective  prob¬ 
abilities  of  correct  response,  thus  necessitating  that  0^  in  the  P^(0)'s 
be  multidimensional.  This  position  is  perhaps  most  clearly  and  vigorously 
pursued  by  Humphreys  (see  Humphreys,  1984).  Humphreys  states: 

"The  related  problems  of  dimensionality  and  bias  of  Items  are 
being  approached  in  an  arbitrary  and  over-simplified  fashion. 

It  should  be  obvious  that  unidimensionality  can  only  be  approxi¬ 
mated.  Even  in  highly  homogeneous  tests  the  mean  correlation 
between  paired  items  is  quite  small.  The  large  amount  of  unique 
variance  in  items  is  not  random  error,  although  it  can  be  called 
error  from  the  point  of  view  of  the  attribute  that  one  is  attemp¬ 
ting  to  measure.  Test  theory  must  cope  with  these  small  correla¬ 
tions.  We  start  with  the  assumption  that  responses  to  items  have 
many  causes  or  determinants." 

Humphreys  (1984)  presents  the  viewpoint  that  dominant  attributes  (dimensions) 
result  from  overlapping  attributes  common  to  many  items.  Attributes  unique 
to  individual  items  or  common  to  relatively  few  items  are  unavoidable  and 
indeed  are  not  detrimental  to  the  measurement  of  dominant  dimensions.  But 
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the  number  of  these  attributes  should  not  be  counted  in  assessing  the 
"dimensionality"  of  the  test.  Humphreys'  writings  stress  that  the  observed 
low  item  intercorrelations  compel  one  to  accept  the  viewpoint  of  multiply 
determined  items.  Unfortunately,  the  classical  definition  takes  as  test 
dimensionality  the  total  number  k  of  all  item  dimensions,  each  item 
requiring  in  general  several  (possibly  many)  dimensions  to  describe  its 
P^(0J.  As  follows  from  the  above  discussion  of  Humphreys'  viewpoint,  this 
fact  is  true  even  in  situations  where  both  from  a  psychometric  and  a  data 
analytic  viewpoint,  one  would  want  to  catagorizc  tests  as  uni  dimensional. 

Thus  the  classical  definition  assigns  dimensionality  d=k>l  (k  thus  assigned  possi 
quite  iarge  in  fact)  in  settings  where  one  would  want  Lo  assign  d  =  i. 

The  following  hypothetical  example  is  intended  to  make  concrete  the  multi¬ 
dimensional  nature  of  items  in  tests  that  should  be  considered  unidimensional. 

Example  5.1.  Consider  a  "probability"  test  where  Item  1  measures  ability  in 
probability  but  is  influenced  by  many  other  idiosyncratic  factors  contribu¬ 
ting  to  "non-error  noise",  e.g.,  a  knowledge  of  the  rules  of  bridge. 

Item  2  measures  ability  in  probability  but  is  influenced  among  other  tilings 
by  the  examinees'  understanding  of  elementary  physics. 

Item  3  measures,  ability  in  probabil ity  but  is  influenced  among  other  things 
by  a  knowledge  of  Mcndelian  genetics. 

One  clearly  is  forced  to  label  such  a  test  as  multidimensional 
according  to  the  classical  psychometric  conceptualization  of  dimensionality 
described  above  (indeed,  clearly  d  >  A  will  be  assigned  with  the  dimen¬ 
sions  including  abiiity  in  probability,  bridge  knowledge,  elementary  physics 
knowledge,  and  knowledge  of  genetics). 
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Clearly  one  wants  a  conceptualization  of  Lest  d Lmensional ity  d 
such  that  tests  such  as  that  of  Example  3.1  would  be  considered  one  dimen¬ 
sional  (i.e.,  d  =  1).  What  is  needed  is  a  conceptualization  of  dimension¬ 
ality  that  does  not  yield  an  inflated  d  as  a  result  of  the  inherent 
mult id imensionality  of  items. 

To  this  end,  the  following  conceptualization  of  dimensionality  is 
proposed.  Consider  a  test  of  length  nd  +  H  administered  to  one  randomly 
selected  examinee  yielding  random  variables  { U ^ ,  1  <_  i  <_  nd  +  M},  where 
n  is  to  be  thought  of  as  possibly  large  compared  to  >1.  Let  (*)  have  a 
k  dimensional  density.  (Here  k  will  likely  In  applications  be  quite 


large  because  of  multiply  determined  items.)  Let 


(5.1) 


z? .  i).  r.n  ..  u. 

1=1  1  v  _  i.=n-H  l  v 

n  ’  2  M  d 


rnd  U 

Ai=n(d-1)+1  i 


define  a  splitting  -of  tlie  test  into  d  "partitioning"  subtest  scores 
and  "assessment"  subtest  score  Y^+^.  bet,  f°r  0  £  y  £  1, 


(5.2)  A  =  A 

y 


yi»  ,,,,yd 


(5.3)  s  =  •• 

Jo 


•1  1 1  f00  r00  2 

...  OilY0  |  B  -  01  -  E[Y2|fl  e  A  1)  f(0|OGA  )dndjy, 

Jn  Jq^-00  j  ~o>  “  ^ 


d  integrals  k integrals 
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where  f(n  [  HGA^)  denotes  the  conditional  dpnsit)  of  given  fiG  A  . 

Relative  to  the  particular  splitting  that  produced  Yj  ,Y7,  . . .  ,  Y  j  ,  s  should 
be  viewed  as  an  Index  measuring  the  lack  ol'  fit  of  the  assumption  that  the 
dimensionality  is  d.  Let  S  denote  the  set  of  the 

nd  +  M 

n  n  ...  n  M 

S - .^r-  ' 

d  n '  s 


possible  such  splittings.  Let 

(5.4)  S  .  =  sup  s  . 
d  s G-S 

Definition  5.1.  A  test  is  said  to  be  of  dimension  d  provided  =  0, 


In  actual  applications,  taking  into  account  the  inherent 
.multidimensionality  of  individual  items,  it  seems  reasonable,  indeed  necessary 
to  assign  dimensionality  d  provided  is  "small"(but  not  necessarily  0)  and 
is  "not  small",  the  quantification  of  "small"  and  "not  small"  varying 
from  application  to  application.  When  this  last  remark  is  taken  into 
account,  Definition  5.1  is  much  in  the  spirit  of  the  factor  analytic 
conceptualization  of  dimensionality  and  is  precisely  in  the  spirit  of  the 
Humphreys' viewpoint  of  dimensionality.  That  is,  the  dimensionality  is 
taken  to  be  the  number  of  common  factors  (dominant  attributes)  with  specific 
factors  (attributes  common  to  relatively  few  items)  not  contributing  to 
the  dimensionality.  The  simple  hypothetical  example  below  is 
intended  to  illustrate  this  admittedly  mathematically  complex  but  actually 
rather  intuitive  definition. 
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Example  5.2.  Definition  5.1  will  be  applied  with  d  =  1.  That  is,  the 
question  whether  d  =  1  in  the  sense  of  Definition  5.1  will  be  addressed. 
Case  1.  Suppose  that  two  types  of  examinees,  "rural"  and  "urban",  take 
a  "reading  test".  Let  the  classical  dimensionality  k  ■  2  with  6^ 
denoting  the  level  of  reading  ability  and  6^  denoting  familiarity  with 
urban  culture.  Suppose,  for  simplicity,  that 

P[0  -  (-1,1)]  =  P[0  =*  (1,-1)]  -  ^  , 
defines  the  latent  ability  space.  Here 


(01,6^)  *  (-1,1)  denotes  low  reading  ability,  high  familiarity 
with  urban  culture 


and 

(0^,02)  ■  (1,-1)  denotes  high  reading  ability,  low  familiarity 
with  urban  culture. 

Consider  the  computation  of  when  d  »  1  in  Definition  5.1.  I.e., 

consider  the  question  of  unidimensionality  in  the  sense  of  Definition  5.1. 
Suppose 


E[YX  |  0  -  (-1,1)]  -  ElYj  0  -  (1,-1)]  *  k 


for  a  particular  splitting  to  obtain  subtest  scores  Y^Y^.  That  is,  both  types  of 
examinees  ((-1,1)  and  (1,-1))  can  be  expected  to  on  the  average  get 
k  of  the  items  on  the  partitioning  subtest  right.  Thus 


-  {(-1,1), (1,-1)}. 

Suppose  that  the  assessment  score  Y9  has  been  formed  from  M  items  with 
identical  item  characteristic  curves,  each  item  dependent  on  0^  only 
(i.e.  the  items  upon  which  Y^  is  based  really  are  pure  "reading"  items. 


24 


In  particular,  suppose 

P^-l)  =  1/4,  I>x(l)  -  3/4 

for  i  =  n  +  l,...,n  +  M  defines  P  (0^).  (The  range  of  0^  is  {-1,1}). 
Since  this  reading  test  is  clearly  contaminated  by  a  second  dimension, 
d  '5  2  should  clearly  be  concluded.  Replacing  the  integrals  by  sums  in 
(."  4)  because  of  the  artificial  discrete  nature  of  this  model,  one  obtains 

(5.5)  Sx  >  s  «  ( K [ Y 2  |  0  =  (-1,1)}  -  -’2)2  ?*  +  (I- [Y2  !  O  =  (1,-1)]  -  ',)\  =  1/16. 

Thus  >  0,  reflecting  the  fact  that  d  >  1  (in  fact  d  =  2)  in  this 
case. 

Case  2.  Now  by  contrast,  consider  a  case  where  reading  ability  is  uncon¬ 
taminated  by  a  second  dimension.  Suppose  in  fact  that  all,  n  +  M  items 
satisfy  for  1  <_  i  £  n  +  M, 

Pi(-1)=  A,,  Pi(0)  -  -'i,  Pi(l)  =  1 

t/here  the  latent  ability  space  (reading  ability  here  denoted  by  0^)  is 
{-1,0,1}.  Here  k  =  1.  Clearly  A,  =  {— 1} ,  A,  =  {0},  and  A.  =  {1} 

'4  -2  -t 

and 


S  *=  sup  s  =  0, 

$ 

leading  one  to  conclude  that  d  =  1,  ns  is  desired  in  this  situation. 

The  above  example  did  not  have  the  classical  dimensionality  k 
large  as  the  result  of  multiply  determined  items.  However,  it  is  intuitively  clear 
that  for  each  test  splitting,  s  as  defined  by  (5.3)  should  be  small  in  Example1  5.1 
since  there  is  only  one  dominant  attribute  there.  For,  conditioning  on 
©  6  Ay  intuitively  amounts  to  conditioning  on  a  subpopulation  of  examinees 
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of  roughly  the  same  probability  ability  (indexed  by  Oj  say)  but  with 
possibly  widely  differing  abilities  on  the  other  k  -  1  components  of 
0.  Thus  for  "typical"  0_'s,  given  0G  A^,  (E[Y^'0=O]  -  E[Y^  |  06  A  ])*" 
should  be  small  since  ^ t JO  =  0^]  should  be  mainly  influenced  by  0^  wjth 
the  effect  of  ^2’’*’’\on  E[Y2!0=OJ  being  small.  Hence,  as  suggested, 
should  be  small  for  situations  like  Example  5.1.  One  can  construct 
examples  showing  that  this  is  indeed  the  case.  The  following  example 
is  such  an  example.  That  is,  the  classical  dimensionality  greatly  exceeds 
1  because  of  multiple  determinants  and  yet  the  dimensionality  according 
to  Definition  5.1  is  1.  This  example  is  highly  artifical  and  is  merely 
intended  to  make  concrete  the  above  comments. 

Example  5.3.  Suppose  the  N  ICC's  are  given  by 
P^(J3)  “  ®1  +  ®i  w^ere  \  ^ 

and  -  <  0^  <  for  i  =  2,...,N.  Suppose  each  0^  is  uniformly 

distributed  over  its  range.  Fix  M.  Split  the  test  into  subtests  of 
size  M,  N  -  M  -  n.  Note  that 

E[Y2|©=  0]  *  01,  E[Yi|0  =  6]  =  01 

Hence 

E[Y2|0<e  Ay]  =  E[Y?  |0^  =  01  and  (©2 ,  • . .  ,©N+1)  unrestricted] 


Thus  recalling  (5.2),  (5.3),  (5.4),  S^  =  s  =  0.  Hence  d  =  1,  even 
though  k  =  N  +  1. 
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Some  final  remarks  in  aiding  the  reader  in  interpreting  (5.2)  - 
(5. A)  seem  in  order.  One  should  consider  d  =  1  for  ease  of  understanding 
and  k  small  as  in  Example  5.2.  Moreover,  one  should  realize  that 
(taking  d  =  1)  the  integration  over  y  ir  merely  an  averaging  process 
over  y  of  the  contribution  to  multidimensionality  resulting  from  each 
choice  of  y  such  that  E[Y^  |  ©  =  _§_ ]  =  y. 

Put  qualitatively,  s  measures,  among  examinees  that  are  expected 
to  have  the  same  score  on  the  partitioning  subtest,  the  amount  of  variation 
there  is  in  their  expected  scores  on  the  assessment  subtest. 


27 


6.  A  discussion  of  power.  The  proceeding  section  suggests  that  if 
(see  (5. A))  is  sufficiently  large,  then  one  would  like  to  reject  the  hypothesis 
of  unidiinensionality  with  high  probability.  The  following  result  establishes 
•ch  a  property  for  long  tests  and  large  examinee  samples. 

Theorem  6.1.  Let  d  =  1  in  (5.3).  Suppose  for  latent  ability 

—  S  ^01,O2’ * ' *  ,(V  that 

(6.1)  the  density  f (_0 )  is  a  continuous  function  of  j), 

(6.2)  each  ICC  P^(0)  is  continuous  in  H > , 

(6.3)  there  exists  c  >  0,  M  1,  and  some  test  splitting  into  subsets 
for  each  n  such  that 

s  >  *  *-C  for  each  n  >  1. 

M  — 

Then,  for  fixed  level  of  significance  a,  and  prescribed  power  3,  0  <  3  <  1, 
(6. A)  Ptreject  H]  =  P[T  >  Z  '  ]  >3 

for  n  sufficiently  large. 

Proof.  See  Appendix  II  for  the  proof. 

The  following  corollary  is  useful  in  Interpreting  psychometrically 

(k) 

the  meaning  of  (6.1).  Let  Y^  -  Y^  ^  be  the  partitioning  subtest  score  of  the 
jth  examinee  of  the  partitioning  interval  A^11^. 

Corollary  6,1.  Assume  all  the  hypotheses  of  Theorem  6.1,  except  for  (6.8). 
Suppose  for  a  particular  test  splitting  that  for  all  n  and  some  c  >  0 
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&  tr  S-i+a  u>  . 


(6.5) 


M) 


•>  c 


J 


'•'hen  (6.3)  holds  and  hence  (6.4)  holds  for  n  sufficiently  large. 


Proof.  See  Appendix  II  for  the  proof. 


The  interpretation  of  the  corollary  is  that  if  the  average  of  the  item 
intercovariances  on  the  assessment  subtest  is  bounded  away  from  0,  then  the  power 
can  be  made  arbitrarily  close  to  1  for  large  n.  Here  the  averaging  is 
over  boLh  subintervals  of  the  partition  and  over  item  pairs  of  the  assessment 
test.  The  idea  is  that  if  conditioning  still  leaves  a  significant  amount  of 
interitem  covariance  on  the  average,  then  one  would  want  to  reject  with 
reasonable  power.  Corollary  6.1  guarantees  this. 
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7.  Preliminary  Monte  Carlo  studios;:  As  ot  the  writing  of  this  report, 
an  extensive  Monte  Carlo  study  has  been  begun.  Its  purpose  is  to  assess 
the  performance  of  the  proposed  statistical  test  of  un  id  imensional  ity  for  values 
,,f  test  length,  examinee  sample  size  and  subtest  splitting  sizes  typically 
encountered  in  actual  test  settings  where  dimensionality  is  an  issue.  A 
part  of  this  study  will  consist  of  the  consideration  of  minor  modifications 
of  the  proposed  procedure  in  order  to  improve  its  performance  for  small  test 
length  N  (and,  as  turns  out  to  he  potentially  useful,  "large"  M/N) . 

Preliminary  Monte  Carlo  studies  clearly  suggest  the  feasibility  of  the 
test  procedure  and/or  minor  modifications  of  it  for  actual  mental  test 
applications.  The  purpose  of  this  section  is  to  summarize  these  preliminary 
findings.  A  Fortran  program  to  do  Monte  Carlo  simulation  of  the  performance 
of  the  test  procedure  (given  by  (3.8))  both  when  d  =  1  and  when  d  *  2 
has  been  written.  Further,  a  method  of  introducing  multiply  determined  items 
has  been  programmed,  thus  handling  the  d  =  2  case  when  the  classical 
dimensionality  k  >  2,  as  well  as  handling  the  k  *  d  =  2  case. 

Briefly,  in  the  d  =  1  case,  the  standard  three  parametric  logistic 
model  is  used  witli  c^  =  1/5  for  all  i  and  parameters  (a.,  b^)  randomly 
generated  to  simulate  values  typically  occurring  in  applications.  Two 
results  obtained  were  that  the  procedure  showed  good  adherence  to  the 
prescribed  level  of  significance  when 

(7.1)  J  -  1000,  N  =  200,  M  =  10 

and  when 

(7.2)  J  =  1000,  N  =  75,  M  =  5. 

Using  a  bivariate  logistic  model  for  the  d  =  2  case,  the  procedure 
displayed  excellent  power  for  the  (7.1)  and  (7.2)  cases. 
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The  introduction  of  multiply  determined  items  produced  only  insignificant 
loss  of  power  and  a  modest  increase  in  the  actual  level  of  significance  a  for 
both  the  (7.1)  and  (7.2)  cases. 

Further,  as  a  result  of  the  preliminary  Monte  Carlo  studies,  there  is 
evidence  that  a  minor  modification  of  the  denominator  of  (3.7),  namely  replacing 


may  considerably  improve  the  performance  of  the  procedure  for  small  test 
length  N,  that  is,  produce  good  adherance  to  the  prescribed  level  of 
significance  with  good  power  performance  still  preserved.  Use  of  this 
denominator  is  suggested  by  the  well  known  Cauchy-Schwarz  inequality  from 
mathematics.  This  and  other  modifications  of  the  proposed  procedure  are 
presently  being  considered. 

As  the  theory  suggests,  preliminary  Monte  Carlo  studies  indicate  the 
accuracy  of  the  large  sample  approximation  begins  to  break  down  when  M/N 
becomes  too  large.  E.g.,  N  «*  100,  M  ®  10,  J  ■  1000  seemed  to  noticeably 
inflate  the  actual  level  of  significance.  Various  possible  modifications  of 

the  procedure  may  largely  eliminate  this  breakdown. 

The  Fortran  coding,  together  with  instructions,  is  obtainable  from 

the  author.  Again,  the  preliminary  nature  of  these  Monte  Carlo  findings 
is  to  be  emphasized. 
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8.  The  choice  of  assessment  Lest'  length  and  assessment:  test  items;  Consider 
a  mental  test  setting  consisting  of  an  N  item  test.  The  choice  of  M  should 
be  "small"  compared  to  test  length  N;  the.  preliminary  Monte  Carlo  studies 
reported  on  in  Section  7  tentatively  suggest  taking  M/N  <  1/10.  On  the 
othe.r  hand,  it  is  also  clear  that  in  a  reasonably  long  test  (e.g.,  N  >_  100), 
the  larger  that  one  can  safely  take  M,  and  thus  the  greater  statistical  power  one 
attains.  In  a  particular  test  application  with  known  N  and  J,  it  would 
be  feasible  for  an  applicator  to  conduct  a  Monte  Carlo  simulation  for  various 
choices  of  M  in  order  to  select  an  M  that  is  large  enough  to  provide  good 
power,  yet  not  so  large  that  the  actual  level  of  significance  deviates  too 
much  from  the  prescribed  level  u. 

An  even  more  critical  issue  is  the  assignment  of  test  items  to  the 

assessment  subtest,  once  M  has  been  determined.  If  done  improperly,  the 

statistical  test  may  have  little  or  no  statistical  power.  The  problem  is 

essentially  that,  even  though  it  may  be  true  that  d  >  1  in  the  sense  of 

Definition  5.1,  nonetheless,  the  assignment  of  assessment  items  may  be  such 
that  the  two  subtests  are  "parallel"  in  a  certain  sense.  That  is, letting 
Yx  and  denote  the  proportions  correct  on  the  partitioning  and  assessment 

subtests  respectively, 

(8.1)  EY2  =  hCEYj) 

may  be  the  case  for  each  examinee,  when  h  is  a  strictly  increasing  function 
that  is  the  same  for  each  examinee.  Then  in  fact  the  statistical  test  will 
have  no  power.  Indeed,  if  the  two  subtests  are  roughly  parallel  in  the 
sense  that  (8.1)  holds  approximately  then  there  will  be  little  or  no  statistical 
power.  Hence,  in  order  that  the  statistical  test  have  reasonable  power,  the 
items  must  be  assigned  to  the  two  subtests  in  such  a  way  that  if  indeed 


d  >  1  is  the  case  then  there  should  be  little  probability  that  the  two 
subtests  will  be  approximately  parallel. 

The  procedure  for  assigning  items  to  the  assessment  subtest  should  be 
such  that,  when  there  are  other  dominant  dimensions  present,  then  the  items 
that  are  more  heavily  influenced  by  these  other  dimensions,  should  (ideally) 
all  be  assigned  to  one  subtest.  And  hence  the  items  assigned  to  the 
other  subtest  should  be  less  heavily  influenced  by  these  other 
dimensions . 

For  the  assignment  of  items  to  the  assessment  subtest,  three  methods 
that  could  be  used  are: 

(a)  Expert  j udgment :  One  or  more  experts  on  mental  testing  could  select  a 
set  of  M  items  that  seem  to  display  a  common  bias  or  seem  to  be 
significantly  influenced  by  a  second  dimension.  This  choice  could  be 
made  on  the  basis  of  item  content,  method  of  item  administration,  item  . 
format,  past  experience  from  previous  test  administration,  subjective 
impressions,  etc. 

(b)  Factor  analytic  or  other  multivariate  analyses:  Split  the  text  examinee 
population  (randomly)  into  two  subpopulations.  Use  some  multivariate 
technique  on  the  first  examinee  subpopulation  to  select  those  items  that 
seem  to  most  represent  other  unified  dimensions  than  the  dominant  test 
dimension.  For  example,  using  the  sample  tetrachoric  correlation  matrix, 
one  could  perform  the  usual  factor  analysis  of  the  test.  Then  those  M 
items  with  the  largest  estimated  positive  loadings  (one  could  use  largest 
negative  loadings  just  as  well)  on  other  dimensions  of  the  factor  analysis 
relative  to  the  dominant  dimension  should  be  assigned  to  the  assessment 
subtest.  Then  carry  out  the  statistical  Lest  of  Section  3  on  the  second 


examinee  subpopulation. 


The  point  to  realize  here  is  that  even  though  there  is  not  an 
adequate  theoretical  or  empirical  basis  to  justify  the  use  of  factor 
analysis  to  assess  the  dimensionality  of  dichotomouslv  scored  items; 
nonetheless,  factor  analysis  can  he  used  as  an  atheoret  ical  datn- 
analytic  tool  in  conjunction  with  the  theoretically  grounded  procedure 
presented  in  this  paper.  Since  often  tesL  population  size  is  quite  large 
in  test  applications,  there  seems  to  be  much  to  recommend  this  approach, 

(c)  Random  assignment  of  items  to  subtests,  subject  to  user  specification 
of  the  magnitude  of  M,n.  The  hope  here  is  that  the  diversity  of 
item  dependence  on  the  various  dimensions  in  the  d  >  1  case  would  with  high 
probability  bring  about  the  selection  of  sufficiently  non-parallel  sub¬ 
tests  that  the  statistical  test  will  display  reasonable  power.  At 
present,  no  theoretical  nor  Monte  Carlo  work  has  been  curried  out  to 
defend  this  hope.  Indeed,  the  author  is  skeptical  of  the  effectiveness 
of  this  method  of  assigning  items  to  subtests,  but  will,  of  course 
investigate  it. 

One  lias  to  choose  the  partition  subintervals.  These  intervals 
should  clearly  be  of  equal  width.  Hence,  the  user  is  really  only 
•selecting  the  number  of  subintervals  Kn<  Roughly,  these  intervals 
should  be  as  narrow  as  possible  subject  to  the  requirements  that  a  reasonable 
number  of  intervals  (e.g. ,  at  least  10)  have  a  reasonable  number  of  examinees  (say 
at  least  25)  ass  igned  to  them  and  t  ha t  not  too  many  examinee  scores  are  d  i scarded . 
One  final  minor  matter  is  that  a  convention  for  excluding  intervals  with  too 
examinees  assigned  to  them  must  be  selected.  E.g.,  one  might  require 
at  least  25  examinees/intervul . 
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After  the  completion  of  the  extensive  Monte  Carlo  study  referred 
to  at  the  beginning  of  Section  7,  the  author  should  be  able  to  provide 
clearer  guidelines  on  the  user  specified  options  discussed  here  in 
Section  8  that  are  required  to  conduct  the  statistical  test. 
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9.  Extension  to  tests  of  higher  dimensionality.  The  procedure  can  be 
modified  to  test 

H:  d  <  dQ  vs  A:  d  >  dQ 

for  any  d^  2.  One  merely  uses  d^  partitioning  subtests  and  one 

assessment  subtest  and  then  partitions  examinees  on  the  basis  of  what 

partitionery  subset  of  the  d^  dimensional  unit  cube  their  partitioning  score 

Y  ,  Y„,...,  Y,  falls  into.  The  author  expects  to  do  some  theoretical  and 
0 

Monte  Carlo  work  on  the  dQ  2  case  in  the  future. 
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10.  Concluding  remarks.  It  should  be  stressed  that  the  main  advantage  of 

the  proposed  procedure  of  this  paper  is  that  it  has  a  rigorous  asymptotic 
theory  backing  it  up.  As  a  result,  it  should  display  good  power  against 
those  alternatives  for  which  one  really  does  wish  to  reject  the  hypothesis 
of  unidimensionality  and  it  should  not  spuriously  reject  unidimensionality 
in  situations  where  one  does  not  wich  to  reject,  as  has  sometimes  been 
the  case  with  other  procedures.  Further,  the  assumptions  required  for  the 
asymptotic  theory  are  rather  unrestrictive  and  are  totally  non-parametric. 
That  is,  no  particular  form,  such  as  multivariate  logistic  or  multivariate 
normal  is  assumed  for  the  item  response  theory  model. 


Appendix  I.  Proofs  of  Theorems  4.1  and  4.2 . 


Let  s<n)  denote  the  radius  of  .  Let  denote 

the  midpoint  of  .  Let  0^  be  defined  by,  writing 

instead  of  0^,  ■  0^  for  the  conditioning  event, 

(X.l)  E[Y<k)|0<n)] 

noting  that  is  well  defined  for  all  1  £  k  £  Kn,  n  1  by  the 

continuity  and  strict  increase  of  the  P^(0)  and  the  definition  of 
ri  in  Section  4.  Suppose,  for  the  e  of  (4.5).  and  (4.7),  that 


(1.2) 


0^  €  (a  +  c,  b  -  e)  for  all  k,n  . 

K. 


1.2 


Recall  that  the  density  of  0  is  denoted  by  f (0) .  The  conditional  pro¬ 
bability  estimate  given  in  Proposition  4.1  below  is  central  to  the  derivnti 
of  the  asymptotic  theory.  It  is  based  on  the  Berry-Esseen  theorem,  which  is 
now  stated  for  completeness.  Let  V(W)  denote  the  variance  of  a  random 
variable  and  $(z)  denote  the  distribution  function  of  a  N(0,1)  random 
variable  throughout. 


Lemma  1.1,  Fix  n.  If  {W^,l  £  i  £  n}  are  independent  mean  0  random  vari¬ 
ables  with 


V(wp  >  0 

then,  letting  S  **  l"  W  ,  F  *  E*?  ,e|w 
°  n  i«l  i  n  i«l  1 

constant  C  such  that 

S 

sup|p[ - ~  ,-fj  <  x]  -  $(x)  |  < 

x  v(s  \Lf £ 

n 

Proposition  1.1.  Fix  k,n  and  denote  sn 


J  ,  there  exists  a  universal 


cr 


n 


v(sn)3/2 

=  s^n),0(n) 


.  Suppose 


(I.  3) 


2n  ^  <  s  < 
—  n  — 


C  n 


-1/2 


Then,  letting  Y=Yp  denote  the  proportion  correct  on  the  partitioning 
subtest  for  a  randomly  chosen  examinee, 


0.4)  p[|e-  e(n)|  >  —  |  v  e  a^o)]  ?-M).  .  - 

s  (n  '  s  -  n  ) 
n  n 

-1  /2 

for  all  s  <  x  <  (log  n)  and  e  of  (4.5)  and  (1.2). 
n  —  — 


Proof .  Clearly 


P[  1©  -  6(n>|>^ |  Y  A"1']  * 


(n),  A 


Pl  |©  -  0^n)  |  _>  — ,  |y  |£  snl 
P(|Y  -  *>(n)  |  £  s  ] 


(1.5) 


1.3 


The  denominator  is  estimated  first.  For  any  y  >  0  , 

(1.6)  P[  | Y  -  v?(n)|  <  sn]  _>  P [  | Y  -  _<  sn  ,  |0-  6(n)| 

f6(n)+y  ,  , 

=  1  P[  |Y  -  |  <  sJ0  =  0]f(0)d9 

e(n)  -y 
g(n)  +y 

>  I  P[l  Y  -  E[Y 1 0]  I  +  -  E[Y 1 0]  I  <  s  |0 

Je(n)-y 

say.  Now  for  0  £  [0^  -y,6^  +y]  , 


|^<n)  -  E[Y 1 0 ] |  =  |E[Y 1 0 (n) ]  -  E[Y 1 0]  | 

z"  i  {p. (0 (n) )  -  P .  (O) } 

=  | -ill - i - - - 1 -  |<  C 1 0 (  ^  -  0 1 


by  the  mean  value  theorem  and  (4.6)  Thus,  continuing  (1.6), 


(1.7) 


e(n)+y 


e(n)-y 


P[|y  -  E[Y|0]|  <  sn  -  Cy|0]f(0)d0 


>-\ 


0(n)+y 

8<n)-y 


P[|Y  -  E[Y|0J|  <  Sn/2|0]f(0)d0  , 


choosing 


(1.8) 


y  1 


2C  * 


Now,  trivially,  writing 

(1.9)  V (Y | 0)  <  n-1  ,  Z”=1E|ui  -  E[U± |0] | 3  <  n  . 


Further,  by  (4.7),  (1.8),  (1.2),  (1.3)  it  follows  that 
(1. 10)  V(nY|0)  -  Pi(9)(l  -  Pi(0))  >  en  for  some  £  > 


1  y] 


«=  0]f  (0  )  dO  = 


Hence,  by  the  Br.rry-Esseen  Theorem  (Lemma  1.1),  noting  (1.9),  (1.10)  and 


(1.3),  it  follows  th.it 


P[|Y  -  E(Y|6)|  <  --  |  6]  ^  ^ — “  |0] 

V(Y  1 0) 


Hence,  using  (1.6)  and  (1.7), 

t 

(1. 11)  P[|Y  -  <  s  1  >  Cs  (s  n1/2  -  n"1/2)  . 

i  —  n  ~  n  n 

Now,  let  x'  »  2x/e  and  consider  the  numerator  of  the  right  hand  side  of 
(1.5)  : 

(1.12)  P[  |Y  -  V>(n)  |  £  sn,  |0  -  0(n)  |  >  x'  ] 

-  f  P{ |y  -  *(n)|  <  s  |0=  0]f(0)d9  . 

V0(n)|>x’  “  " 

Assume 

(1.13)  [0  -  0(n) |  >  x\a  <  0  <  b  . 

Now 

(1.14)  6  =  P[|Y  -  E[Y|0(n)]  |  <  sn|0]  <  P[|Y  -  E[YlO(n)]  |  <  x|0] 

£  P[ |Y  -  E[Y|0) |  >  |E[Y|0]  -  E[Y|0(n))|  -x 

since,  on  [  | Y  -  E[Y|u(n)]|  <  x]  , 

x  >  |Y  -  E(Y|0(n)]|  >  |E[Y|0]  -  E[Y|0(n)]|  -  |Y  -  E[Y 1 0]  |  . 

Now,  using  (4.5)  and  (1.2),  it  follows  that 

|E[Y|G]  -  E[Y|0(n))|  _>  £  f  0  -  0(n)|  >cx'  =  2x 


1.5 


Hence,  using  (1.12)  and  (1.14) 

(1.15)  6  <  P[|Y  -  E[Y j 6] |  >  x j 0] 


^urE[uil°)| 
v(Ei=1ui|0)1/2  ~ 


_ nx _ 

v(£i-1u1l°)1/2 


0 


Now, 


lyEiuJeii  t 

v(nYje)1^2  ~  va"  u  |e>1/2 


Let 

£i  „  nx _ 

v(zj=1ui|0)1/2 

Note  that 

(1.16)  e'c'  «  - — -  0  as  n  ->-«> 

v(Ei=1uil0) 

-1/2 

by  (4.7)  and  the  hypothesis  that  x  <  (log  n)  .  But,  this  means  that 
conditional  on  ©  ■  0  ,  the  satisfy  the  conditions  of  the  classical 

Kolmogorov  exponential  bounds  (see,  e.g. ,  Stout  (1974),  p, 262).  That  is, 

2 

6  <  exp  (1  -  e'c')] 

from  which  it  follows  by  (1.16)  and  (1.9)  for  n  large  that  for  0  such 
that  1 0  -  0(n)|  >  x\  0  e  (a,b)  , 

(1.17)  P[|Y  -  E[Y|6(n)j|  <  sn|©  =  0]  <  exp(:~1)  . 


Now  assume  instead  that  6  satisfies 
(1.18)  |9  -  6(n)|  >  x’  ,  0  *  (a,b) . 

Then;  for  0  >  b  ,  using  the  mean  value  theorem  and  (4.5), 


1.6 


E[Y|0]  -  E[Y|0(n)]  >  E [ Y  |  b  J  -  E[Y|G(n)]  >  £2  . 

Thus,  using  a  similar  argument  for  0  <  a  , 

E[Y I e ]  -  E[Y|0(n)] I  >  e2  . 


Hence,  reasoning  as  in  (1.14),  it  follows,  for  n  large  that 
6  <  P(  | Y  -  E[Y 1 0]  |  _>  E2/2|0  =  0]  . 

Hence,  for  any  e  *►  0  ,  for  n  large 

(1.19)  6  £  P[  jY  -  E[Y 1 0]  j  >  £n|©  =  8  ]. 


Now,  it  is  straightforward  to  verify  (see  Lamperti  (1966),  p.  44))  for 
0  <  t  <  2  that 


(1.20)  E[exp(t  n(Y  -  E[Y|0])|0]  <  exp 


VlPi(0)(1  '  Pl(0»  2 

- - 1 -  t\l  +  t) 


Then,  by  Markov’s  inequality,  for  0  <  t  <  2  ,  . 
(1.21)  P[n|Y  -  E[Y  1 0 ]  |  _>  n  1 0 ]  2  exp(-n  t)exp 


O  t  \ 


There  are  two  cases  to  consider.  First,  suppose  that  n  is  such  that 


(1.22)  2J»1Pi(0)(l  "  Pi<0>>  >  2n/log  n  . 


Then  (see  (7),  p.  44  of  Lamperti  (1966))  choosing 


t  - 


2  e  n 


,  e  = 


Ei=lPi(0)(1  '  pi(0))  ”  108  0 


yields 

S  <  P[n|Y  -  E[Y|e)|  0]  <2  1 


(1.23) 


Suppose,  instead  of  (1.22)  that 


1.7 


Ii-ipi(0>(i  -  pi(0»  <  isfir 


Then,  letting  t  ■  •g  and  using  (1.21)  yields 


(1.24)  6  <  P[n|Y  -  E [Y 1 6]  1  |0]  <  2  exp( 


Wi(9>^-Pi< 

8  log  n  ex^  64 


<  2  exp  (77-: - )  . 

—  r  16  log  n 


Combining  (1.21)  and  (1.24)  yields,  recalling  (1.14), 


(1.25)  6  =  P[|Y  -  E(Y|9(n))|  <  sn|0]  <  2  exp(Irg__) 


for  0  satisfying  (1.18).  Recalling  (1.17),  it  follows  that 

6  <  moJexp(^),2e*p<— n> 

2 

„  „  ,-x  ns 

<  2  exp (  ^  ) 

for  0  satisfying  |0  -  0^|  j>  x'  by  the  assumption  that  x  _<  (log  n) 
Hence,  u&lng  (1.12), 

P[|y  ~  *(n)|  <  sn,|0  -  0(n)|  >  x’]  <  2  exp (-^~) . 

Thus,  recalling  (I. 11)  and  (1,5)  ,  the  desired  result  follows. 


Let  k  s  denote  a  sequence  of  positive  integers  throughout  the 
remainder  of  Section  4.  Recall  that 


(1.26) 


(k)  E  Zn+M 
j  i=n+l  i j  k 


/M 


defines  the  assessment  subtest  score  for  examinee  .j  of  the  partitioning 
interval  can  easily  he  shown  (and  is  intuitively  clear)  that 

(1.27)  {yf^ }  is  identically  distributed  in  j  for  fixed  k  and  independent 

J 

in  j ,  k  . 


The  proof,  which  follows  from  (2.3)  and  (2. 4),  is  omitted.  Let 


1.8 


Y.  n  be  the  partitioning  subtest  score  of  the  jth  examinee  of  the 
J  I*  . . 


partitioning  interval  A 


(n) 


(1.28) 


Bj  =  Bj,k  “  lYj,P  e  Ak  )],C1i 


Recalling  the  notation  of  (2.1),  let 
(n) 


ai';k  =  P[Ui+n,lk“ 


and 


(1.29) 


c  =  c  =  c 
n 


n,  k 


=  Ii=1  ai(l  -  ai)/M2. 


(k) 

Denote  the  density  of  0.  given  B.  by  f(«)=f,(*).  Let 

i  i  »  n  n 

J  =  throughout. 

In  order  to  verify  that  the  asymptotic  distribution  of 
a2  a2  a2  a2 

°Y  “  °P  "  °y  k  “  °P  k  ^aS  mean  *s  necessary  to  consider  the  pre- 

asymptotic  centering:  . 


Proposition  1.2.  Suppose  (1.3)  and  that  uniformly  in  k(l  £  k£  K^) 

(1.30)  ■+  o  as  n 
n 

2  2  (k)i 

Then,  letting  °y  =  °y  n  ^  =  V(Yj  j  B ^ ,  it  follows  uniformly  in  k  that 

(1.31)  J^2(<Jy2  -  c  )  ■*  0  as  n+»  . 

Suppose  (1.3)  and,  letting  K  =  Kn,  that  uniformly  in  k, 

(1.32)  '  J*<lof  n)*  -  0  as  „*»  . 

n 

Then,  uniformly  in  k  , 

(1.33)  (JK)1/2(oy2  -  c  )  0  asn-M»  . 


Proof.  Only  the  proof  of  (1.33)  will  be  given,  since  the  proof  of  (1.31 )  is  vir¬ 


tually  identical.  First,  0^=  E(V(Y  1 0) )  +  V(E(Y  1 0) )  .  Thus 

Oy2  =  r^U+l  Jpi<0>  C1  -  P1(0))f<n)(0)d0/M2' 

, ,  rn+M  /  v  rn-tM  \  o 

+|[(Iizn±i!i^ '  f<")(0)do 


denoting 


,00 


J  — 0 


1.9 


liZr  “t  h> 


r  n+M 


1 


M 

frn+M 


rn+M  _  ,A.  rn+M  \ 
£i=n+lPi(0)  Sl-„+l  “l 


M 


M 


£(n)(0)dO  . 


Now, 


r+M  pi(6) 

i=n+l 

rn+M 

I  “i 

i=n+l 

2 

teLv°>] 

M 

M 

M 

2  \n+M  /A.  »n+M 

“  2  U=n+lPi(0)  ^l=n+l  ai 

M 


’  I  yn+M  a 

H=n+1  i 
a,  +  ' - -a — 


M 


C+1P?<8>  .  „I).:IilI'<„+MPl(e>Pi'(9) 


2  rn+M  n  /n.rn+M 

„2  ■  m2  "  h2  ^.„+lPl(0>ii.n+l  ai 


rn+M 


rn+M  2  V 

^i=n+l  ai  n+l_<l^i 1  <.n+M  ajai' 

*■  2  ’  ^  • 


M 


M 


Thus, 


<  - «  - 1 

n+l<i^i  <n+M 


E[P1(01)Pi,(01)|B1] 


M 


o 

1  (rn+M  1  1  rn+M  2 

-7lli-n+l“ij  +-2Wlni 

"  J  LkW<n«*!'Pl*ei*Pl'*ei^,l'  aial,J  ' 

M  —  •  — 


Thus,  it  suffices  to  show  that,  uniformly  in  n  +  l£i^i'  _<n  +  M, 

,1/2 


(JK)X/Z  cov(Pi(01),l>it(01)|B1)  -*■  0  . 


Hence,  it  suffices  to  show 

(jk)1/2v(p1(©1)  |bx)  +  0 

as  n  -*>  00  ,  uniformly  in  n  +  1  £  i  <  n  +  M. 


Now,  letting  x'  =  2x/e 


1.10 


(1.34)  v(p1(0l)|b1)  =  j[P.(0)  -  P.(e')f(n)(O')dO']'2f(n)(0)clO 


■ 

3(n,+»' 

P.  (0)  -  [p . (0’)f (n>(0,)d0t 

2f(n)(0)d0 

6<n)-x' 

J  1 

+ 

► 

fp. (0)  -  fp. (0*)f (n> (0*)d0’ 

0(n)-x' 

,8(nVlcL  ' 

J 

f(n)(O)d0 


=  R  +  S  ,  say  . 


Now,  by  Proposition  1.1,  for  any  £  x  <  (log  n) 


-1/2 


(1.35)  S  <  £ 


,  172 - -177  =  Vsay 

sn  n  s„  '  "  > 


Consider  R.  Then,  using  the  mean  value  theorem,  there  exist  0^  satisfying 


|6  -0^n^|  <_  x'  such  that 


(1.36)  |Pt(6)  -  |pi(e')f<n>(0,)de,| 

|pi(e>  -  U,  -1 

I 


)+x'  (e*>f Cn>(e,)de'- J 


[e(n)-x',  e(n)+x’)c 


P1(e,)t!  °( 


e!"!+x'  p  (0’)'f(n)(0,)d0* 

9'n'_v*  * 

p  (0)  _  * - 

4  'e(n)+*'  . _x 


fo (n)  V">~- 

J  /„*  fW(0')d0'  0 


|0V  '+x'  f(«)(9.)d9. 


0(n)-x’ 


+  Q 


n 


V9>  -  Pi(Vt(n)!x!£<n)(9')d9’  +9n> 


P^©) 


£nV  !£<">(e,)de'  +  \ 


f(n)(0')d0' 


£  |p±<0)  -  P1(e0)|{ 


-  pi(9o)|“ll)^£<",'9'>d9' 


+  Q. 


0^n^+x'  fn) 

,s  fW(0)d0  +  2  Q  <  Cx'  +  2  Qn  , 
0  n  -x' 


1. 11 


by  the  mean  value  theorem  and  (1.34).  Thus,’  by  (1.34)',  (1.35),  and  (1.3C)) 

(1.37)  (JK)1/2V(P  (0  )!b  )<[(Cx'  +  Q  )2  +  q1(jK)1/2  . 

1  L.  *1  iij 

Thus,  it  suffices  to  obtain 

(x')2(JK)1/2  -  0  ,  Qn(JK)1/2  -*■  0  . 

1  /2 

Recall  that  x'  ■  2 x/c.  Taking  x'  =  (log  n)/n  works  for  example,  estub- 

i 

lishing  the  result. 


Next,  the  asymptotic  distribution  of 
malized,  is  obtained. 


suitably  nor- 


Propositton  1.3.  Suppose  (1.3).  Suppose  for  some  c>0  and  all 
n  +  1  <  i  <  n  +  M,  n  ^  1  that  (recalling  that  0^  S  oj^) 

(1.38)  |Pi(0(n))  -  1/2 1  >  e  . 


Suppose  (1.30).  Let 


(1.39)  D  =  D 


=  n<n>  = 


(2«x  -  1,  2a2  -  !,•••, 20^  -  1,1) 


and  ,  denoting  E(Y  -  E(Y))  by  y^(Y), 


(1.40)  E  =  Z, 


(n) 


a1(l-a1) 


a2(l-a2) 


0 

0 


ait(1-aM) 


a1(l-a)(l-2a1) 


a2(l-a2)(l-2a2 


4  (  a) 

a1(l-a1)(l-2a1)  a2(l-a2)(l-2a2)“-a:iU-aM)(l-2aM)  M  [n/((Y)-0vj 


Yj 


1.12 


Then,  letting  Z  denote  a  N(0,1)  •  random  variable, 


£  2  *  2 

2  _  °v  ”  t 

(1.41)  H  - T7o  Z  as  n->-“  , uniformly  in  k  . 


(DZD’)1/2 


^  o  ^2 

Proof.  Since  writing  "o^  ~  Op"  presupposes  the  assignment  of  examinees 


by  the  partitioning  subtest,  it  is  implicit  in  the  statement  of  Propositior 

J 


1.3  that  the  conclusion  of  (1.41)  is  conditional  on  O  B  . 

3  =  1  ^ 


,(k) 


Let.  writing  ^  =  Yj  Uy  5  Un+1>jk  ,nnd  LYj  =  F.[Yj|B  ], 


w!  -  <W-.uMj  ,  1C(T  -ET  )*) 


Note  that 


2_  2 


(1.42)  E[W*  1|B1]  =  (2a1-li2a2-l,»**,2aM-l  Hoy')  . 


We  next  compute  the  asymptotic  behavior  of  the  covariance  matrix  of  W 

given  Bj .  Note  that,  by  conditioning  on  0^ , 

(1.43)  oi  -  E(Uil|B1)  -  E[Pn+i(©1)|B1) 

and  for  i  ^  i' 

E(Uil-Ul'llV  '  E[Pn+l(ei)Pn+i'(9l)lBJ  ■ 

Recalling  the  proof  of  Proposition  1.2,  for  i  j  i'  ,  it  follows  that 

(1.45)  J1/2  c°v(Pn+i  (0X)  ,Pn+i » (0X)  iBp)  -  0 
and 

j1/2  V(Pn+i(0i)  |B  )  ->  0  as  n->°°  . 

Thus,  combining  (1.43),  (1.44),  and  (1.45),  it  follows  that 
1/2 

(1.46)  J  cov^Uil’Ui'l  J B1)  ->  0  as  n  ■>»  . 

Now,  by  (4.7)  and  (1.2),  for  some  e  >  0  and  all  I,k,n 
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(1.47)  1  -  e  >  Pi(G(n))(l  -  Pi(0(n)) )  >  c  . 

Now,  for  x^  =  (log  n)  \  using  (4.4)  , 

foo 

(Pi(0)  -  P.(e(n)))J-(n)(y)dO| 

■"  j-  CO 

<  P[ |0,  -  Q(n) j  >  X  |b. ]  +  Cx 
—  1  1  n '  1  n 

Hence,  applying  Proposition  1.1,  it  follows  that 

(1.48)  ai  -  P1(G(n))  •*  0  ns  n-*°° 

uniformly  in  i  .  Thus,  combining  (1.47)  and  (1.48),  and  recalling  (1.29), 

(l.49)  inf  c  >  0  ,  1  -  e  >  a ,  (1-a  )  >  e  for  all  i  . 
n>l  n 

Note  that 

(1.50)  V(UiilV  =  "  ot-l  . 

.Further,  using  (1.46)  and  elementary  calculation ,  denoting  an^n">  ^  '’Y  an  " 

(1.51)  cov(Uil,M2(Y1-EY1)2|B1)  -  £((1^-0^) 3 ll^]  =  0^(1  -u^l  -  2a.)  ds  n->' 

Thus,  recalling  (1.40)  and  (1.46),  (1.50),  and  (1.51),  it  has  been  establishe 
that,  termwise,  as  n->-«>  , 

(X.52)  V£. 

Also,  note  by  (1.38)  and  (1.48),  it  follows  that,  for  some  e  >  0 
( I i 53)  1 2a  ^  -  l|  >  e  ,  |ai(l-ai)(l-2ai)|  >  e 


uniformly  in  i  . 


1.14 


Now,  armed  with  the  above  computations,  the  proof  proceeds 
essentially  by  a  multivariate  version  of  the  "  5  method"  (see  p.  122, 
Serfling  (I960)).  Let 

(1.54)  *g(x1,x2,  =  xml  -  E^=1  x.(l-Xi) 


Thus 

2x^-1  i  <  H 

1  i  ■  M+l 

Let,  recalling  (1.42), 


r,w/J 


j=i  J 


The  main  thrust  of  the  proof  is  the  establishment  of  the  asymptotic  normality 
of  g(Xj)  • 

Let 


h(x) 


fg(x)  -  g(yj)  -  g(jjj  ;  x  -  bj) 


o 


if  X  t  v J 

if  £  =  lij 


where 


(1.55)  g(y,t)  -  |^~ 


t  +  .  •  .  4-  - 

1  9xIl+l 


'M+l  ’ 


the  differential  of  g  at  (£»t_)  and  Hx  !l  ‘  2  .  ^ « 

x^  .  Let 


bj  “  ^T72(d  1  D')1/2 


Now, 


8<Xj)  “  g(lij) 


htxp- 


-  Hi 


8^J  ’  -J  “  ^ 


(1.56) 


1.15 


By  (1.55),  letting  denote  the  ith  'component  of  jiJt 

d.57)  v .  i  rx,  -  ,,tJ>  . 


im 


J1^2(D  Z  D')1//2  i=l 


>1 


Ji 


(l!^  -ai)(2a.-l)  +  MCY^EY  )2-  a*) 


J1/2(D  Z  d')1/2  ‘-1 


H=1 


'1=1  i.l 


-ai)(2cxi-l)  +  M2((Y1-EYj)2-  oY2) 


Let 


=  R]k>  =  Zi=l<Uij  -ct1)  (2m1-l )  +  M2((Yj(k)-KYj(k))2  -  a2  ). 


Now,  using  (1.52), 

.  J 

(1.58)  V(R  |  n  B,()  =  V  ( R  |  B .  )  ~  (D  Z  D'). 

j  j.=1  J  2  J 

Moreover,  by  (2.3)  and  (2.4),  it  is  easy  to  show  that  the  {lO  are,  for 
each  n  ,  i.i.d.  By  the  Berry  Esseen  theorem,  it  suffices  to  show  that 

2 

inf  >  0, 
n  1 

which  will  follow  from 

(1.59)  inf  (D  l  D')1/2  >  0 

n 

Since  D  is  asymptotically  bounded  away  from  the  M  +  l  dimensional  vecto 
0  ,  it  suffices  to  verify  that  asymptotically  Z  is  bounded  away  from  the 
collection  of  (M+1)*(M+1)  singular  matrices.  To  this  end,  it  is  neces- 

4 

sary  to  obtain  an  asymptotic  expression  for  y^(Y)  -  <7^  .  Computation  sho 


that 


1.16 


V400  -  ay4  ~  +  6ai  "  3ai} 


+  2  E  a  .(1-a  )a  ,  (1-a  ,) 
1<1^KM  111  1 


rM  2  ,,  .2 

-  £t.i  “id"0!)  • 


Then,  although  mildly  tedious,  using  |a^(l-ai)|  >  £  for  all  i  ,  some 
£  >  0  ,  it  can  be  shown  that  E  is  indeed  bounded  away  from  the  collec¬ 
tion  of  singular  matrices.  Thus,  noting  (1.57)  and  applying  the  Lindeberg- 

J 

Feller  theorem,  conditional  on 

(1.60)  ±  eCHjJXj  -  Pj)  X  Z  . 

J 

where  Z  is  a  N(0,1)  random  variable.  Now,  since  the  elements  of 

1/2  d 

E/J  approach  0  as  n  ■*■<*>  ,  it  follows  that,  conditional  on  » 


X j  -  ]J j  — *■  0  as  n  -►» 


Hence,  conditional  on  .H  B  , 

j-1  j 


h(Xjj  -  h(jij)  —*■  0  as  n 


by  the  continuity  of  h  at  ]Jj  .  But,  since  Mjij) 

J 

conditional  on  jOiBj  » 


0  ,  it  follows  that, 


h(Xj)  — ►  0  as  n-*-00  . 


Hence,  referring  to  (1.56),  to  obtain  asymptotic  normality,  it  suffices  to 


show  that 


—  >  x  Ln^Bj  ->-0  as  x-oo , 


1.17 


uniformly  in  n  .  Hence,  by  Chebychev's  inequalit  it  suffices  that 


DID*  is  bounded  below,  which  is  known  from  (1.59).  Thus,  conditional 

J 

on  »  letting  Z  denote  a  N(0,1)  random  variable, 


(I.  61) 


g(Xj)  -  b(Hj) 


Z  as  n->°° 


Note  Chat 


(1.62) 


J1/2(oy2-  oy2,  -  J1'2 


£j-i(YrEYi)2 


- 


-  J1/2(Y  -  EY;[)2 


Recall  (1.49)  and  (1.31).  Hence  inf  >0.  Thus 

,3  n 


n  E(|Y1-EY1| 


.3/2  2 

°Y 


-  1/2  ‘ 
n 


Hence  by  the  Berry-Esseen  theorem  (Lemma  1.1)  it  follows  that,  conditional 
on  jPjBj  . 

1/2  —  T 

(1.63)  J  '  (Y-EY^)  4  Z  ,  a  N(0,1)  random  variable. 

J 

By  Chebychev's  'inequality,-  conditional  on  .O  B, 

3  "■  J 


(1.64)  Y  -  EY1  0  . 


w 

Thus,  (1-63)  and  (1.64)  together  imply  that,  conditional  on 


1/2  —  2  p 

J  '  (Y  -  EYX)  -£•  0  . 

Thus,  referring  to  (1.62)  and  (1.54),  it  follows  that  replacing  E^(Yj-EYj) 

2 

by  Oy  in  (1.61)  leaves  the  conclusion  of  (1.61)  unchanged.  Applying 
(1.31)  of  Proposition  1.2  to  (1.61)  and  recalling  (1.54)  then  yields  the 
desired  conclusion  for  each  fixed  k  =  k  sequence.  Although  not  explicitly 


1.18 


stated,  all  arguments  In  the  above  segment  of  the  proof  hold  uniformly 
in  k,  thus  establishing  the  desired  conclusion. 

Now  the  proof  of  Theorem  4.1  can  be  completed. 


Proof  of  Theorem  4.1.  Trivially  follows  from  Proposition  1.3  and  the  facts  that 
J 

conditional  on  n  B., 

j-1  J 

(1.65)  cij.  ~  ai  ■B">  0,  y^(Y)  -  u4(Y)  0,  0  as  n  ■+•  °° 

J 

uniformly  in  i,k.  Also  note  that  conditioning  on  O  B  is  implicit  in 

j=l 

the  statement  of  Theorem  4.1. 

Remark.  Note  that,  recalling  (3.6)  and  (3.7) 

(i* 66)  (fi£6')  -  M4(y4(Y)  -  a4>k))  -  %  . 

Now,  the  proof  of  Theorem  4.2  can  be  given. 

4n) 

/  \  k  k  /  \ 

Proof  of  Theorem  ,4.2.  Note  that  condition  on  Bv‘  Bn  n  b'  ,  is  implicit 

k*l  j-1 

in  the  statement  of  Theorem  4.2.  The  proof  basically  consists  of  a  modification 
of  the  proof  of  Proposition  1.3.  The  main  steps  are  presented,  with  the  details 
left  to  the  interested  reader:  Refer  to  the  proof  pf  Proposition  1.3.  Let 


-  (U 


U  U  /y(W  py(k)\2v 

n+1,  jk,Un+2,jk* '  *  ’  *  n+M,  jk’  j  j  }  *' 


As  in  the  derivation  of  (1.52),  it  follows  that  termwise,  as  n  -+•  00 , 


(I-67>  l  (k>  '  k 


(n) 


w 


uniformly  on  k.  Define  g(*)  by  (1.54).  Let 


T(n) 

„  rJk  „(k)/T(n)  „  K 

"  k-  •  ^ 

n  k=l 


r  n 
^k-l 
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Note  that 


*<v 


i  K 

1  r  n 
K  Ik=l 

n 


T(n) 

r  y k 


(yfk)  -  eyJr)) 

■  ^  —  -  J  —  - 


r(n) 


rM  *00,.  -(k) 

Ik-1  Pi  (1  "  pi 


Define  g(M>t)  and  h(x)  as  in  the  proof  of  Proposition  1.3.  Let 


K 


.2  r”n  (n)r(n)  (n) ' /.(n) 

bn  ■  4-1  Dk  4  \  /Jk 


The  rest  of  the  proof  proceeds  analogously  to  that  of  Proposition  1.3,  noting 


that  it  is  necessary  to  use  (1.33)  in  place  of  (1.31).  One  concludes, 


conditional  on  B 


(n) 


that 


(1.68)  M 


K 

Ik-i  « 


Y,k  ~  °P,k 


'  L><n)  £<n)  D<n)i)/j<n)11/2 


z. 


Now,  conditional  on 


g(n)  s(n)  . ^(n)',  ,(n) 
Pk  \  Dk  /Jk 

D<n)  Z<n)  D<n)'/J<n) 


as 


uniformly  in  k  follows  from  .the  fact  that  (1.59)  and  (1 .65)  hold  uniformly 
in  k.  Thus,  noting  again. that  (1.59)  holds  uniformly  in  k,  conditional 
on  B(n), 


K 


(1.69) 


n  £(n)  £(n)  g(n),,J(n) 
k  ^k  uk  /Jk 


y n  & 

Ik=l  UV 


K 


1  as 


n  -*■  °°. 


Ik-i  4n)4n)  <n)'^n) 


Combining  (1.68)  and  (1.69)  yields  the  desired  result. 


II. 1 


Appendix  II.  Proof  of  Theorem  6,1  and  Corollary  6.1. 

Proof  of  Theorem  6.1.  Note  that  conditioning  on  is  implicit  on  the 

statement  of  (6.4).  Let  k  denote  the  classical  test  dimensionality  and 

If 

R  denote  k  dimensional  Euclidean  space.  It  follows  trivially  from 
Chebychev's  inequality  that  the  weak  law  of  large  numbers  holds  for  the 
partitioning  test  score  Y^,  uniformly  in  0..  That  is, 

(II.  1)  -  E[Yl  |o  =  £]  *-*■  0  as  n 

where  the  convergence  in  probability  is  uniform  in  6.  G  r  ,  jt  thus  follows, 
denoting  the  distribution  function  of  a  random  variable  W  by  F  and 

w 

using  (6.2),  uniformly  in  £  and  y  that 


(II. 2)  F^(y|0-  0)  -  Fe| 


(y  |  0  =  £)  -»•  0  as  n  °° 


EIYJ0] 

for  each  0  <  y  <  1,  Then,  for  fixed  y,  it  is  easy  to  show,  using  (6.2 ) 
and  denoting  the  conditional  density  of  0,  given  Y  ■  y,  by  f(»  |  y)  and’ 
the  conditional  density  of  0,  given  E[Y  |  0]  ■  y  by  g(»  |  y)  that 


Jr" 


Z?,!  Pj.(0>  f  Ii=i  Pj (O') 


1  2 


M 


•I. 


k  N 


f(0'|  y)d0’ 


f(0  |  y)d0 


< 

f  4-1  f  4-1  pl(-'>  g(6'|  y)d6* 

'V  M  "  JRk  « 


g(0  [  y )d0  *►  0 


as  n  %  uniformly  in  y.  Thus,  integrating  over  y,  it  follows  that 


(II. 3) 


lira 

n-*» 


Pi (O') 

M 


2 


f(0'  |  y)d0' 


f(0  |  y)d0 


dy 


2 


g(6'  |  y)d0 


g(0  |  y)d£  dy  0 


as  n  -*■  ». 


11.2 


Similarly , 


<“■*>  Pn  -  «n  P  f  j„k 

n  ;  R 


*  f 


fi-1  Pl<S> 


M 


-I, 


*  -  f(i'  I  Yx  e  A^n))10 


M 


f1  f 

1 1.' 


Cl  v-°->  I  fl-i  pi<°’>  .... 


f(£|  Yx  6  A<n))dO 


M 


-I, 


1  2 


M 


f(0'  |  y)dQ' 


f  (0  |  y)d0^  dy 


as  n  °°. 


Combining  (II. 31,  (II. 4),  and  using  (6.3),  it  thus  follows  that 


(II. 5)  lim  P 
n-*°° 


>  1  ±  c/2 


n  M 


It  is  easy  to  see  that,  recalling  (1.28)  and  (1.29), 


(II. 6) 


E(c£  u  |  B(n)) 


Y,k 


E(°P,kJB<n)) 


^  -  1 
,(n) 


(°Y,k  ‘  Cn,k^’ 


Arguing  as  in  the  proof  of  Proposition  1.2, 


(II. 7)  K  Ir-1  ^°Y,k  "  cn,k*  “  •  ..2  Ik=l  ^i-1  I  Y1  6  Ak  ^  +  Pn 


KnM 


>  P  “  77. 

—  n  M 


Using  (II.  5),  it  thus  follows  that 


(II. 8)  lim  K  Jk-1  (0y^k  -  cn>R)  >  2M  . 


n-w  n 


II. 3 


Thus,  using  (II. 6)  and  (II. 7),  for  all  n  sufficiently  large,  conditional  on 
it  follows  that 


(11.9,  i  E(S2_k  |  B<n))  -  E(Op>k  |  B<">)  >  I;  . 


Now, 
(II. 10) 


(n) 


I‘|T  >  Z  I  bv -  P 
u  a  1 


_1_  r  n  (%2  -2 

K„  Jk-1  '  Y,k  ‘  “p.k1 

-  ^  {E(o‘  |  B'"')  -  E(3^  |  B(n))} 

rt  ^  B 


.2  ZafrKn  (g(n)!(n)S(n)'/J(n)|1/2 

K,  K  K  K  I 


„  L  (  K. 

*  <  {«. 


-  f  Iw{Et0?.l 


|B(n)) 


*2 

-  e(o; 


>k|B(n))}| 


,  (n) 


But,  trivially 


1/2 


n 


<  C  -»•  0 
—  n 


as  n  «.  Thus,  using  this  and  (II. 9)  and  applying  Chebychev's  inequality, 

for  large  n 


plTn  >  Za  J  B^  ]  >  1  -  ~  1  as  n 


Thus,  the  result  is  proved. 


II. 4 


Proof  of  Corollary  6.1.  Note  that  conditioning  on  Ye  is 

1  I  i  K 

implicit  in  the  statement  of  Corollary  6.1.  It  is  easy  to  show  that 

I  C0V(U  ,U  ,  I  Ye  A<n)) 
ia  hM  *  »r  K 


l<i<i’<M 


at  i  “  c  .  *  2 
Y,k  n,k 


M 


ii-i  v<p±<gl  h,p6  Akn)> 


M 


Hence,  (6.5  )  implies  that 


11m 

n-*» 


til  (°Y.k  * 


n 


Cn,k)  £ 

-  2  ’ 


a  result  analogous  to  (11,8).  The  rest  of  the  proof  proceeds  identically  to 
that  of  Theorem  6.1  and  is  omitted. 
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Alexandria,  VA  22333 

2  Mr.'  Robert  Ross 


Air  Force 

1  Cr.  Alfred  F: .  Frsgly 


1  Dr.  Genevieve  Haddad 
Program  Mareger 
Life  Sciences  Directcrate 
AFOSR 

Bolling  AFS,  D3  20332 

1  Dr.  Patrick  hyilorsn 
AFHRL/MDE 

Brooks  AFB,  T<  75235 

1  Lt.  Col.  D=ye  Payne 
AFHRL/MDE 

Brocks  AFB,  TX  73235 

1  Dr.  Roger  Fennell 
Air  Force  Hunan  Resources  Laboratory 
Lowry  AFB,  CC  80230 

1  Dr.  Mai  col  2  Fee 
AFHSL/NP 

Brooks  AFB,  TX  78235 

1  Major  John  Helslr 
AFHRL/MQAN 

Brooks  ACB  ,  TX  73223 


U.S.  Arcy  Research  Institute  for  the 
Social  and  Behavioral  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 


1  Dr.  Robert  Sasitor 
U.  S.  Amy  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Joyce  Shields 
Amy  Resea*ch  Institute  for  the 
H*  Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
'  Alexandria,  VA  22333 

1  Cr.  Hilda  king 
Aray  Research  Institute 
5001  Eisenhower  Ave. 

Alexandria,  VA  22333 


Illinois/Stout 
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De?art.T.s  .t  of  Defense 

12  Defense  Technical  Isforsaiicn  Center 
Caseron  Station,  21dg  5 
Alexandria,  VA  22314 
Attn:  TC 

4  1  Military  Assistant  for  Training  and 
,  Personnel  Tecnnology 

•»  Office  of  the  Under  Secretary  of  Defers 

for  Feaearch  4  Engineering 
Poor.  3D129,  The  Pentagon 
Washington,  DC  20301 

1  Dr.  W.  Steve  Selloan 
Office  of  the  Assistant  Secretary 
of  Defense  (HR A  4  L) 

2B269  The  Pentagon 
Washington,  DC  20301 

1  Dr.  Robert  A.  Wisher 
OUSDRE  (ELS) 

The  Pentagon,  Room  3D129 
Washington,  DC  20301 


Cw: lien  Agencies 

1  !>.  Vsrr.  N.  ‘Jrry 
Personnel  FiD  Center 
Office  of  Personnel  hanagecsnt 
1900  E  Street 

lv3S -i i  i  Vw  kv4.j 

1  hr.  Th.on.a5  A.  Warn 
U.  S.  Coast  Susrd  Institute 
P.  0.  Substation  13 
CHahoaa  City,  DK  73169 

1  Dr.  Joseph  L.  Young,  Director 
Memory  4  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20350 


Illinois/Stout 
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Fr i vats  Sector 

1  Dr.  Jsir.es  Algina 
University  of  Flcriis 
Gainesville,  FL  32i05 

1  Dr.  Erlir.;  B.  A'isr =en 
Ceoartsent  of  state  sties 
-  Stodiestraede  6 

J  1455  Copenhagen 
.  DENMARK 

1  Dr.  Isaac  Eejar 
Educational  Testing  Service 
Princeton,  NJ  03450 

1  Dr.  Menucha  Eirenbaue 
School  of  Education 
Tel  Aviv  University 
Tel  Aviv,  Rasat  Aviv  fc??73 
Israel 

1  Dr.  R.  Darrell  Boci: 

Departnent  of  Education 
University  of  Chicago 
Chicago,  IL  60637 

% 

'  1  Dr.  Robert  Erennan 
•  American  College  Testing  Progress 
P.  0.  Bos  168 
Iowa  City,  I A  52243 

1  Dr.  Glenn  Bryan 

6203  Poe  Road 
Belhesoa,  KB  20317 

1  Dr.  John  B.  Carroll 
40?  Elliott  Rd. 

Chapel  Hill , * NC  27514 

1  Dr.  Noraan  Cliff 
Dept,  of  Psychology 
Uni v.  of  So.  California 
University  Park 
Los  Angeles,  CA  7)007 

1  Dr,-  Hans  Crcabag 
Education  Research  Center 
v  University  of  Leyden 
Soerhaavelaan  2 
„  2334  EN  Leyden 

The  NETHERLANDS 

1  CTB/KcG'aw-Hill  Library 
2500  Garden  Road 
Monterey,  CA  93940 


Private  Sector 

I  rr.  kalter  Cunning'-:;--1 
L'r.iversit*  of  “ :  a t : 
Dspartaent  of  Fayeh.olcgv 
Eainesvilis,  FL  32c 11 

1  Dr.  Dattcradad  Divgi 
Syracuse  University 
Cepartne.-it  of  Psychology 
Syracuse,  NS  332i0 

1  Dr.  Ecnancsl  Dor. chin 
Department  of  rsycnolocy 
University  of  Illinois 
Chaapaign,  IL  6IG20 

1  Dr.  Hei-Ki  Done 
Ball  Fcj.icstion 
Race  314,  Building  B 
000  Roosevelt  Road 

•  Glen  Ellyn,  IL  62137 

1  Dr.  Frit:  Drasgow 
Department  of  Psychology 
University  of  Illinois 
603  E.  Daniel  St. 
Chaapaign,  IL  elB20 

1  Dr.  Susan  Eifcertson 
PSYCHOLOGY  DEPARTMENT 
UNIVERSITY  OF  KANSAS 
Lawrence,  KS  66045 


-  — -  - ,  - - - - - 

4333  Rugby  Avenue 
Bethesda,  KB  20014 

1  Dr.  Bsnjasin  A.  Fairbank,  Jr. 
M:Fanr.-Sray  a  Associates,  Inc. 
5025  Callaghan 
Suite  225 

San  Antonie,  TX  78225 

1  Dr.  Leonard  Feldt 
Lindouist  Center  for  “easursent 
University  of  Iowa 
Iowa  City,  IA  52242 

1  Uni v.  Prof.  Dr.  Gerhard  Fischer 
Liebigoasss  5/3 
A  1010  Vienna 
AUSTRIA 


Illinois/Stout 
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Private  Sector 

1  Pro{esscr  Dc.nald  Fitrgs^aid 
lini\er'ity  cf  Ns«  England 
Arsidaie,  New  South  Wales  1151 
AUSTRALIA 

,  1  Dr.  Janice  Gifford 

University  of  Massachusetts 

*  School  of  Education 
Asherst,  HA  01002 

1  Dr.  Robert  Glaser 

Learning  Research  l-  Developeent  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
PITTSBURGH,  PA  15260 

1  Dr.  flarvin  D.  Slock 
217  Stone  Hall 
Cornell  University 
Ithaca,  NY  14353 

1  Dr.  Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  l  34th  Street 
Baltieore,  HD  21218 

m 

•  1  DR.  JfiMES  S.  GBEENO 
"  LRDC 

UNIVERSITY  CF  PITTSBURGH 
3939  O’HARA  STREET 
PITTSBURGH,  PA  15213 

I  Dr.  Ron  Hanbietcn 
School  of  Education 
University  of  Massachusetts 
Aiher st,  HA  01002 

1  Dr.  Paul  Horst 
677  G  Street,  *184 
Chula  Vista,  CA  90010 

1  Dr.  Lloyd  Humphreys 
Department  of  Psychology 
,  University  of  Illinois 

603  East  Daniel  Street 
Chanpaign,  !L  61320 

N 

1  Dr.  Steven  Hunka 
'«  Department  of  Education 
University  of  Alberta 
Edmonton,  Alberta 
CANADA 


Private  Sector 

1  Dr.  Earl  Punt 
Dept,  of  Psychology 
University  of  Hashing  ter 
Seattle.  WA  9S105 

1  Dr.  Huynh  Hu'/nh 
College  of  Education 
University  of  South  Carolina 
Columbia,  SC  29203 

1  Dr.  Douglas  H.  Janes 
Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Court 
Lawrenceville,  NJ  03143 

1  Dr.  Marcel  Just 
Department  of  Psychology 
Carnegie-Nellon  University 
Pittsburgh,  PA  15213 

1  Dr.  Denetrios  Karis 
Department  of  Psychology 
University  of  Illinois 
603  E.  Daniel  Street 
Chaapaign,  1L  61G20 

1  Professor  John  A.  Keats 
Department  cf  Psychology 
The  University  of  Newcastle 
N.S.K.,  2308 
AUSTRALIA 

1  Dr.  Willi  a*  Koch 
University  of  Texas-Austin 
Measurement  and  Evaluation  Center 
Austin,  TX  73703 

1  Dr,  Alan  Lesgclri 

Learning  PJD  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
Pittsburgh,  PA  15260 

1  Dr.  tUensei  Levine 

Department  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Chaapaign,  IL  6180] 


ill inoi s/Stout 
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Private  r  'ter 

1  Dr.  Dairies  Lewis 
Facul te: t  Socials  Wetenschappen 
Rijksuniversiteit  Groningen 
Dude  Bcterirgestraat  23 
97123C  Broningsn 
Netherlands 

•> 

J 1  Dr.  Robert  Linn 

*  College  of  Education 
University  of  Illinois 
Urbana,  IL  618C1 

1  Mr.  Phillip  Livingston 
Systems  and  Applied  Sciences  Ccrporatio 
6811  Kenilworth  Avenue 
Riverdale,  MD  20340 

1  Dr.  Robert  Lockman 
Center  for  Naval  Analysis 
200  North  Beauregard  St. 

Alexandria,  VA  22311 

1  Dr.  Frederic  H.  Lord 
Educational  Testing  Service 
’  Princeton,  KJ  C3541 

*  1  Lf .  J«C.eS  LuUSueii 

*  Departeent  of  Psychology 
University  of  Western  Australia- 
Nedlands  It.  A.  6009 

AUSTRALIA 

1  Dr.  Don  Lyon  • 

P.  b.  Box  44 
Higley  ,  AZ  85236 

1  Dr.  Scott  Maxwell 
Department  of  Psychology 
University  of  Metre  Daae 
Notre  Bane,  IN  46556 

1  Dr.  Saatel  T.  Mayo 
Loyola  University  of  Chicago 
820  North  Michigan  Avenue 

'  Chicago,  IL  60611 

1  Mr.  Robert  McKinley 
American  College  Testing  Programs 
u  P.0.  Box  168 

Iowa  City,  IA  52243 


Private  Sector 

i  Dr.  Barbara  Means 
Hunan  Resources  Research  Organisation 
300  Nortn  Gaining ton 
Alexandria,  VA  22:14 

1  Dr.  Rote't  Mislevy 
711  Illinois  Street 
C.  ueva,  11.  60124 

1  Dr.  in.  Alan  Nicewander 
University  of  Oklahoma 
Department  of  rs/cholcgy 
Oklahoma  City,  OK  73069 

1  Dr.  Donald  A  Norman 
Cognitive  Science,  C-015 
Uni v.  of  California,  San  Diego 

La  Jolla,  CA  92093  ’ 

1  Dr •  Janes  Cl  son 
SICAT,  Ire. 

1875  South  State  Street 
Ores,  UT  84057 

1  Wayne  M.  Patience 
American  Council  on  Education 

t*  c ..?*■«  on 

- -  5  *  «  *  ‘  *  J  - *  *  “  ‘ 

One  Dupont  Cirle,  NM 
Washington,  DC  20036 

1  Dr.  Jajae.5  A.  Paulson 
Portland  State  University 
P.0.  Bex  75! 

Portland,  DR  9-7207 

1  Dr.  James  S.  Pellegrino 
University  of  California, 

Santa  Barbara 
Dept,  of  Psychology 
Santa  Barabara  ,  CA  93106 

1  Dr.  Steven  E.  Poi track 
Bel!  Laboratories  2C-444 
tOO  Mountain  Ave. 

Murray  hill,  NJ  07974 

1  Dr.  Mark  D.  Reckase 
ACT 

P.  0.  fox  168 
Iowa  City,  IA  52242 


1 1! inoi 5/Stout 
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Private  Sectc*- 

1  Dr.  Thomas  Reynolds 
University  of  Te  as-Dallas 
Marketing  Department 
P.  0.  Box  BBS 
Richardson,  TX  75020 

•i 

1  Dr.  Laurence  Pudner 
453  Els  fl/enue 
Takcca  Park,  MD  20012 

1  Dr.  J.  Ryan 
Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29203 

]'  PROF,  FUHIK3  SAKEJIBA 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  TENNESSEE 
KNOXVILLE,  TN  37916 

1  Dr.  Halter  Schneider 
Psychology  Departaent 
603  E.  Daniel 
Champaign,  IL  61320 

1  Lowell  Schoer 

csl  ?i  Susnti 1 yp 

*  Foundations 

*  College  of  Education 
University  of  Iowa 
Iowa  City,  Ift  32242 

.  y.  Kacu:  Shigsiasu 

7-9-24  Kugenusa-Kaigan 
Fujusawa  251 
JAPAN  ’ 

1  Dr.  Nilliafl  Siss 
Center  for  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  H.  Wallace  Sinai Ico 
Frogra.il  Director 

Manpower  Research  and  Advisory  Services 

Saithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

*  1  Martha  Stocking 

Educational  Testing  Service 
Princeton,  NJ  CE541 


Private  Sector 

1  Dr.  Peter  Stolcff 
Center  for  Naval  Anal  .sis 
20  j  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  Hariharan  Sifaunacban 
Laboratory  of  Fsycncoeiric  ar.S 
Evaluation  Roses rc.n 
School  of  Ecucsticn 
university  of  Msssach-setts 
Asherst,  MA  01003 

1  Dr.  Kikusi  Tats.oka 
Computer  PaEed  Education  Research  Lab 
252  Engineering  Research  Laboratory 
Urban*’.  IL  61201 

1  Dr.  Maurice  Tatsuoka 
220  Education  Bldg 
1310  S.  Sixth  St. 

Chaspaign,  IL  61320 

1  Dr.  David  Thissen 
Department  of  Psychology 
Uni ver5ity  of  Kansas 
Lawrence,  K£  66044 

1  Dr.  Douglas  Towne 
Univ.  of  So.  California 
Behavioral  Technology  Labs 
1045  S.  Elena  Ave. 

Redondo  Beach,  CA  90277 

1  Dr.  Rcosrt  TsutaxanS 
Department  of  Statistics 
University  of  Missouri 
Columbia,  MO  65201 

1  Dr.  Ledyard  Tucker 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  Street 
Chaspaign,  IL  61820 

1  Dr.  V.  R.  S.  Uopuluri 

Union  Carbide  Corporation 
Nuclear  Division 
P.  0.  Box  Y 
Dak  Ridge,  TN  37330 


13-!pr  24 
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Private  Setter 


1  Dr.  David  Vale 
A -=5S5 ''snt  Systens  Corporation 
2233  Uni  varsity  ft ver.ua 
Suite  313 

St.  Paul,  NX  55114 


«  1  Dr.  Howard  “ainer 

'*  Division  of  Psychological  Studies 

•  Educational  Testing  Service 
Princeton,  KJ  0E540 

1  Dr.  Michael  T.  Haller 
Department  of  Educational  Psychology 
University  of  Hiscrnsin — Ril waukee 
•Milwaukee,  «I  53201 

1  Dr.  Brian  Haters 
HuaRRG 

300  North  Washington 
Alexandria,  Vfl  22314 

1  Dr.  David  J.  Hales 
N460  Elliott  Hall 
University  of  Minnesota 

•  75  E.  River  Feed 

*■  Minneapolis,  MN  55455 

• 

•«  1  Dr.  Rand  R.  Hi  Icon 

University  of  Southern  California 
Department  of  Psychology 
Los  Angeles,  Cfi  90007 

2  Sr;  Srucs  Hill  isos 

Dep3rtffisnt  of  Educational  Psychology 
University  of  Illinois 
Urbana,  IL  61301 

1  Ms.  Marilyn  Wingersky 
Educational  Testing  Service 
Princeton,  NJ  0C341 

1  Dr.  Her.dy  Yen 
CTB/Xc5r«  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 


